Information processing device and method

ABSTRACT

The present technology relates to an information processing device and a method capable of adaptively providing a partial image of an image in each layer of an image constituted by a plurality of layers. The information processing device of the present technology generates a file that stores information on a whole of a base layer of encoded data produced by layer encoding of image data, information on each of partial areas of an image of the base layer, information on a whole of an enhancement layer of the encoded data, and information on each of partial areas of an image of the enhancement layer such that each of the information is stored in corresponding one of tracks different from each other. The present technology is applicable to an information processing device, an image processing device, an image encoding device, or an image decoding device, for example.

CROSS REFERENCE TO PRIOR APPLICATION

This application is a National Stage Patent Application of PCTInternational Patent Application No. PCT/JP2015/067234 (filed on Jun.16, 2015) under 35 U.S.C. § 371, which claims priority to JapanesePatent Application No. 2014-135147 (filed on Jun. 30, 2014), which areall hereby incorporated by reference in their entirety.

TECHNICAL FIELD

The present technology relates to an information processing device and amethod, and more particularly to an information processing device and amethod capable of adaptively providing a partial image of an image ineach layer of an image constituted by a plurality of layers.

BACKGROUND ART

As a content distribution technology utilizing hypertext transferprotocol (HTTP), moving picture experts group—dynamic adaptive streamingover HTTP (MPEG-DASH) has been standardized in recent years (forexample, see Non-patent Document 1). This MPEG-DASH adopts an adaptivebitrate streaming (ABS) technology which stores a plurality of encodeddata in a content server as data representing identical content bydifferent bitrates, and allows a client to select one of the pluralityof encoded data in accordance with a network band and reproduce theselected data.

Incidentally, it has been considered to adaptively select a partialimage corresponding to a part of an entire image and distribute theselected partial image, rather than distribute the entire image. Forexample, it has been considered to distribute a partial image selectedfrom an entire image by an image data receiving terminal, and control asize of a partial image to be distributed in accordance with performanceof a terminal (such as processing capability of CPU and size ofdisplay), a transmission channel, a server loading state, and otherconditions.

In addition, it has been considered to adopt layer encoding toefficiently encode an image constituted by a plurality of layers byusing prediction between layers or the like, for example.

CITATION LIST Non-Patent Document

[Non-Patent Document 1] MPEG-DASH(Dynamic Adaptive Streaming over HTTP)(URL:http://mpeg.chiariglione.org/standards/mpeg-dash/media-presentation-description-and-segment-formats/text-isoiec-23009-12012-dam-1)

SUMMARY OF THE INVENTION Problems to be Solved by the Invention

However, conventional MPEG-DASH standards only have a concept ofswitching bitrates, and therefore is difficult to achieve adaptiveprovision of partial image data, i.e., adaptive provision by selectingan arbitrary partial image in an arbitrary layer of an image constitutedby a plurality of layers, and providing data on the selected partialimage while utilizing tile structures of the respective layers.

The present technology has been proposed in consideration of theaforementioned circumstances. An object of the present technology is torealize adaptive provision of a partial image of an image in each layerof an image constituted by a plurality of layers.

Solutions to Problems

An aspect of the present technology is directed to an informationprocessing device comprising a file generation unit that generates afile that stores information on a whole of a base layer of encoded dataproduced by layer encoding of image data, information on each of partialareas of an image of the base layer, information on a whole of anenhancement layer of the encoded data, and information on each ofpartial areas of an image of the enhancement layer such that each of theinformation is stored in corresponding one of tracks different from eachother.

The file generation unit may store, in the track storing the informationon the whole of the base layer, encoding information indicating anencoding system of the base layer. The file generation unit may store,in each of the tracks storing the information on the partial areas ofthe base layer, encoding information indicating that the correspondingtrack stores only the information on the corresponding partial area ofthe base layer. The file generation unit may store, in the track storingthe information on the whole of the enhancement layer, encodinginformation indicating an encoding system of the enhancement layer. Thefile generation unit may store, in each of the tracks storing theinformation on the partial areas of the enhancement layer, encodinginformation indicating that the corresponding track stores only theinformation on the corresponding partial area of the enhancement layer.

The file generation unit may store, in the track storing the informationon the whole of the base layer, information indicating reference to thetracks storing the information on the partial areas of the base layer,and information indicating a type of the reference. The file generationunit may store, in each of the tracks storing the information on thepartial areas of the base layer, information indicating reference to thetrack storing the information on the whole of the base layer, andinformation indicating a type of the reference. The file generation unitmay store, in the track storing the information on the whole of theenhancement layer, information indicating reference to the track storingthe information on the whole of the base layer and informationindicating a type of the reference, and information indicating referenceto the tracks storing the information on the partial areas of theenhancement layer and information indicating a type of the reference.The file generation unit may store, in each of the tracks storinginformation on the partial areas of the enhancement layer, informationindicating reference to the track storing the information on the wholeof the enhancement layer, and information indicating a type of thereference.

The file generation unit may further store, in each of the tracksstoring the information on the partial areas of the enhancement layer,information indicating reference to the track storing the information onthe partial area of the base layer in correspondence with thecorresponding partial area of the enhancement layer, and informationindicating a type of the reference.

The file generation unit may store, in each of the tracks storing theinformation on the whole of the layers, a sample of referenceinformation for the encoded data on the partial areas of thecorresponding layer. The file generation unit may store, in each of thetracks storing the information on the partial areas of the layers, asample of the encoded data on the corresponding partial area.

The file generation unit may store, in each of the tracks storing theinformation on the partial areas of the layers, position informationindicating a position of the corresponding partial area.

The file generation unit may generate the file for each of the tracks.

The file generation unit may generate the file for each of the layers.

The file generation unit may generate a file storing all of the tracks.

An aspect of the present technology is directed to an informationprocessing method generating a file that stores information on a wholeof a base layer of encoded data produced by layer encoding of imagedata, information on each of partial areas of an image of the baselayer, information on a whole of an enhancement layer of the encodeddata, and information on each of partial areas of an image of theenhancement layer such that each of the information is stored incorresponding one of tracks different from each other.

A different aspect of the present technology is directed to aninformation processing device comprising a metadata generation unit thatgenerates metadata that contains information on a whole of a base layerof encoded data produced by layer encoding of image data, information oneach of partial areas of an image of the base layer, information on awhole of an enhancement layer of the encoded data, and information oneach of partial areas of an image of the enhancement layer.

The information storing information on the whole of the base layer maycontain encoding information indicating an encoding system of the baselayer. The information on each of the partial areas of the base layermay contain encoding information indicating that only the information onthe corresponding partial area of the base layer is contained. Theinformation on the whole of the enhancement layer may contain encodinginformation indicating an encoding system of the enhancement layer. Theinformation on each of the partial areas of the enhancement layer maycontain encoding information indicating that only the information on thecorresponding partial area of the enhancement layer is contained.

The information on the whole of the base layer may contain informationindicating reference to the information on the partial areas of the baselayer, and information indicating a type of the reference. Theinformation on each of the partial areas of the base layer may containinformation indicating reference to the information on the whole of thebase layer, and information indicating a type of the reference. Theinformation on the whole of the enhancement layer may containinformation indicating reference to the information on the whole of thebase layer and information indicating a type of the reference, andinformation indicating reference to the information on the partial areasof the enhancement layer and information indicating a type of thereference. The information on each of the partial areas of theenhancement layers may contain information indicating reference to theinformation on the whole of the enhancement layer, and informationindicating a type of the reference.

The information on each of the partial areas of the enhancement area mayfurther contain information indicating reference to the information onthe partial area of the base layer in correspondence with thecorresponding partial area of the enhancement layer, and informationindicating a type of the reference.

Each of the information on the partial areas of the base layer, and theinformation on the partial areas of the enhancement layer may containposition information indicating a position of the corresponding partialarea.

The metadata generation unit may set, in the information on the whole ofthe base layer, a file that stores a sample of reference information forthe encoded data of the partial areas of the base layer. The metadatageneration unit may set, in the information on each of the partial areasof the base layer, a file that stores a sample of the encoded data ofthe corresponding partial area of the base layer. The metadatageneration unit may set, in the information on the whole of theenhancement layer, a file that stores a sample of reference informationfor the encoded data of the partial areas of the enhancement layer. Themetadata generation unit may set, in the information on each of thepartial areas of the enhancement layer, a file that stores a sample ofthe encoded data of the corresponding partial area of the enhancementlayer.

The metadata generation unit may set a file that stores the encoded dataof the base layer in an order higher than the information on the wholeof the base layer and the information on the partial areas of the baselayer. The metadata generation unit may set a file that stores theencoded data of the enhancement layer in an order higher than theinformation on the whole of the enhancement layer and the information onthe partial areas of the enhancement layer.

The metadata generation unit may set a file that stores the encoded dataof all of the layers in an order higher than the information on thewhole of the base layer, the information on the partial areas of thebase layer, the information on the whole of the enhancement layer, andthe information on the partial areas of the enhancement layer.

The metadata generation unit may set the information on the whole of thebase layer, the information on the partial areas of an image of the baselayer, the information on the whole of the enhancement layer, and theinformation on the partial areas of an image of the enhancement layersuch that each of the information is set in corresponding one ofadaptation sets different from each other, or in corresponding one ofsub representations different from each other.

A different aspect of the present technology is directed to aninformation processing method generating metadata that containsinformation on a whole of a base layer of encoded data produced by layerencoding of image data, information on each of partial areas of an imageof the base layer, information on a whole of an enhancement layer of theencoded data, and information on each of partial areas of an image ofthe enhancement layer.

According co the aspect of the present technology, generated is a filethat stores information on a whole of a base layer of encoded dataproduced by layer encoding of image data, information on each of partialareas of an image of the base layer, information on a whole of anenhancement layer of the encoded data, and information on each ofpartial areas of an image of the enhancement layer such that each of theinformation is stored in corresponding one of tracks different from eachother.

According to the different aspect of the present technology, generatedis metadata that contains information on a whole of a base layer ofencoded data produced by layer encoding of image data, information oneach of partial areas of an image of the base layer, information on awhole of an enhancement layer of the encoded data, and information oneach of partial areas of an image of the enhancement layer.

Effects of the Invention

According to the present technology, image processing is achievable. Inaddition, according to the present technology, adaptive provision of apartial image of an image in each layer of an image constituted by aplurality of layers is achievable.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a view illustrating an outline of MPEG-DASH.

FIG. 2 is a view illustrating a configuration example of MPD.

FIG. 3 is a view illustrating time segmentation of content.

FIG. 4 is a view illustrating an example of a layer structure in anorder of Period and lower in MPD.

FIG. 5 is a view illustrating a configuration example of a MPD file on atime axis.

FIG. 6 is a view illustrating examples of bit streams of tile images.

FIG. 7 is a view illustrating an example of MP4 files of tile images.

FIG. 8 is a view illustrating a different example of MP4 files of tileimages.

FIG. 9 is a view illustrating a configuration example of an MP4 fileformat.

FIG. 10 is a view illustrating an example of a division size.

FIG. 11 is a view illustrating an example of extended data.

FIG. 12 is a view illustrating an example of layer switching.

FIG. 13 is a view illustrating an example of layer switching.

FIG. 14 is a block diagram illustrating a general configuration exampleof a file generation device.

FIG. 15 is a view illustrating a general configuration example of MP4files.

FIG. 16 is a view illustrating the general configuration example of MP4files.

FIG. 17 is a flowchart showing an example of a flow of an MP4 filegeneration process.

FIG. 18 is a view illustrating a general configuration example of MPD.

FIG. 19 is a view illustrating a general configuration example of MPD.

FIG. 20 is a flowchart showing an example of a flow of an MPD generationprocess.

FIG. 21 is a flowchart showing the example of the flow of the MPDgeneration process, continuing from FIG. 20.

FIG. 22 is a block diagram illustrating a general configuration exampleof a file reproduction device.

FIG. 23 is a flowchart showing an example of a flow of an MP4 filereproduction process.

FIG. 24 is a flowchart showing an example of a flow of an MPDreproduction process.

FIG. 25 is a view illustrating a different configuration example of MP4files.

FIG. 26 is a view illustrating the different configuration example ofMP4 files.

FIG. 27 is a flowchart showing a different example of the flow of theMP4 file generation process.

FIG. 28 is a view illustrating a different example of MPD.

FIG. 29 is a view illustrating the different example of MPD.

FIG. 30 is a flowchart showing a different example of a flow of the MPDgeneration process.

FIG. 31 is a flowchart showing the different example of the flow of theMPD generation process, continuing from FIG. 30.

FIG. 32 is a view illustrating a further configuration example of MP4files.

FIG. 33 is a view illustrating the further configuration example of MP4files.

FIG. 34 is a flowchart showing a further example of the flow of the MP4file generation process.

FIG. 35 is a view illustrating a further configuration example of MPD.

FIG. 36 is a view illustrating the further configuration example of MPD.

FIG. 37 is a flowchart showing a further example of the flow of the MPDgeneration process.

FIG. 38 is a flowchart showing a different example of the flow of theMP4 file reproduction process.

FIG. 39 is a flowchart showing a different example of the flow of theMPD reproduction process.

FIG. 40 is a view illustrating a still further example of MP4 files.

FIG. 41 is a view illustrating the still further example of MP4 files.

FIG. 42 is a flowchart showing a still further example of the flow ofthe MP4 file generation process.

FIG. 43 is a view illustrating a still further example of MPD.

FIG. 44 is a view illustrating the still further example of MPD.

FIG. 45 is a flowchart showing a still further example of the flow ofthe MPD generation process.

FIG. 46 is a flowchart showing a further example of the flow of the MP4file reproduction process.

FIG. 47 is a flowchart showing a further example of the flow of the MPDreproduction process.

FIG. 48 is a block diagram illustrating a general configuration exampleof a distribution system.

FIG. 49 is a block diagram illustrating a general configuration exampleof a computer.

MODE FOR CARRYING OUT THE INVENTION

Modes for carrying out the present disclosure (hereinafter referred toas embodiments) are described hereinbelow. Note that the respectiveembodiments are described in the following order.

1. First Embodiment (filing per tile)

2. Second Embodiment (reference between layers per tile)

3. Third Embodiment (filing per layer)

4. Fourth Embodiment (filing all layers)

5. Fifth Embodiment (distribution system)

6. Sixth Embodiment (computer)

1. First Embodiment

<Dash>

As a content distribution technology utilizing hypertext transferprotocol (HTTP), there has been known moving picture expertsgroup—dynamic adaptive streaming over HTTP (MPEG-DASH) as described inNon-patent Document 1, for example. This MPEG-DASH adopts an adaptivebitrate streaming (ABS) technology which stores a plurality of encodeddata in a content server as data representing identical content bydifferent bitrates, and allows a client to select one of the pluralityof encoded data in accordance with a network band and reproduce theselected data.

Content transfer procedures according to DASH are hereinafter describedwith reference to FIG. 1. Initially, software for streaming data controlincluded in a content receiving side moving image reproduction terminalselects a media presentation description (MPD) file of desired content,and acquires the selected file from a web server. MPD is metadata whichmanages distributed content such as moving images and voices.

The software for streaming data control of the moving image reproductionterminal having received the MPD analyzes the MPD, and performs controlto acquire data (DASH segments) satisfying quality of a communicationline, performance of the moving image reproduction terminal, and otherconditions of the desired content from the web server. Client softwarefor HTTP access acquires the DASH segments from the web server underthis control by using HTTP. Moving image reproduction softwarereproduces the content thus acquired.

MPD has a configuration illustrated in FIG. 2, for example. For analysis(parse) of MPD, a client selects an optimum from attributes ofrepresentations (Representations) contained in periods (Periods) of theMPD (Media Presentation in FIG. 2).

The client reads an initial segment (Segment) of the selectedrepresentation (Representation) to acquire an initialization segment(Initialization Segment), and processes the acquired initializationsegment (Initialization Segment). Thereafter, the client acquires asubsequent segment (Segment), and reproduces the subsequent segment(Segment).

Note that a relationship between a period (Period), a representation(Representation), and a segment (Segment) is expressed as illustrated inFIG. 3. More specifically, one media content is managed for each period(Period) corresponding a unit of data in a time direction, while eachperiod (Period) is managed for each segment (Segment) corresponding to aunit of data in a time direction. In addition, each period (Period) maybe constituted by a plurality of representations (Representations)having different attributes such as bitrates.

Accordingly, a file of the MPD (also referred to as MPD file) has alayer structure illustrated in FIG. 4 in an order of period (Period) andlower. FIG. 5 further illustrates an example of the structure of the MPDarranged on a time axis. As apparent from the example illustrated inFIG. 5, a plurality of representations (Representations) are present foran identical segment (Segment). The client adaptively selects any one ofthe representations to acquire appropriate stream data in accordancewith a communication environment, self-capacity of decoding and otherconditions, and reproduces the acquired stream data.

<Tile Structure>

Conventional DASH as described above adaptively controls distribution ofdata contained in an entire image. However, it has been considered toadaptively select and distribute a partial image corresponding to a partof an entire image, rather than distribute the entire image. Forexample, it has been considered to distribute a partial image selectedfrom an entire image by an image data receiving terminal, and control asize of a partial image to be distributed in accordance with performanceof a terminal (such as processing capability of CPU and size ofdisplay), a transmission channel, a server loading state, and otherconditions.

For realizing adaptive distribution of a partial image as describedabove, a concept of a tile has been adopted. A tile is a partial areacorresponding to a division of an entire image divided in apredetermined layout (such as size, shape, and number). An image of atile is hereinafter referred to as a tile image. When an entire image isdivided into tiles beforehand as described above, adaptive distributionof a partial image is easily realizable only by selecting a tile imageto be distributed. In this case, a partial image is constituted by asingle or a plurality of tile images.

For distribution by utilizing HTTP as in the case of DASH, image data isencoded, whereafter bit streams of the encoded data are filed anddistributed (published as files). In case of an entire image having atile structure as described above, image data is encoded independentlyfor each tile image. In this case, encoded data for each tile may beconstituted by one bit stream as in an example illustrated in A of FIG.6.

According to the example in A of FIG. 6, a 640×480 size entire image, a1980×1080 size entire image, and 960×540 size tile images correspondingto two divisions of the latter entire image in a vertical direction andtwo divisions of the latter entire image in a horizontal direction (fourpartial images) are prepared as distribution images. Data in the 640×480size entire image is encoded into one bit stream (bitstream1), whiledata in the 1980×1080 size entire image is also encoded into one bitstream (bitstream2). Apart from these data, data in each of the 960×540size tile images is encoded independently from each other into one bitstream (bitstream3 through bitstream6).

Header information such as video parameter set (VPS), sequence parameterset (SPS), supplemental enhancement information (SEI), and pictureparameter set (PPS) is added to each of the bit streams. The respectivebit streams of the image data are arranged for each slice (Slice).

This structure allows selection of a tile image to be distributed basedon selection of a bit stream to be distributed from bitstream3 throughbitstream6. Moreover, the structure of the example in A of FIG. 6 allowsdistribution of each tile image in a manner similar to distribution ofan entire image.

Incidentally, an encoding system such as high efficiency video coding(HEVC) supports a structure called tiles dividing an entire image, andperforms encoding independently for each tile. For example, decoding maybe performed in a manner acquiring only an image of a tile correspondingto a part of the entire image. In other words, decoding may be performedin a manner acquiring only a partial image corresponding to a part ofthe entire image.

By utilizing a function of this encoding system, encoded datacorresponding to a plurality of tile images may be collected into onebit stream (bitstream7) as illustrated in an example in B of FIG. 6.More specifically, a tile to be distributed as described above isencoded while handled as a tile supported by the encoding system. Inthis case, data corresponding to respective tiles are arranged as slices(Slices) in the bit stream.

<MP4 File>

As described above, a bit stream to be distributed is filed in an MP4file format or the like, for example. In this case, bit streams ofrespective tiles may be filed in different files for each as in anexample illustrated in FIG. 7. A bit stream of each tile is managed inunits of track. In addition, header (Header) information on each tile,and a base track (Base track) describing reference to each track areprovided, and filed in a file different from the bit streams of therespective tiles. For decoding all tiles, the base track is reproduced.For decoding tiles for each, the base track is referred to for headerinformation.

As in an example illustrated in FIG. 8, bit streams of respective tilesmay be collected into one file. In this case, respective datacorresponding to the respective tiles may be collected and managed asone track as illustrated in A of FIG. 8, or may be managed as differenttracks for each as illustrated in B of FIG. 8. In this case, header(Header) information for each tile, and a base track (Base Track)describing reference to each track are provided similarly to the caseillustrated in FIG. 7.

<MP4 File Format>

An outline of an MP4 file format is hereinafter described. Asillustrated in FIG. 9, an MP4 file in conformity to MPEG-DASH containsftyp, moov, and mdat.

As illustrated in FIG. 9, data on each sample (picture) of HEVC isstored in mdat as AV data.

In addition, management information is stored in a sample table box(Sample Table Box (stbl)) for each sample (such as picture).

As illustrated in FIG. 9, the sample table box (Sample Table Box)contains a sample description box (Sample Description Box), a time tosample box (Time To Sample Box), a sample size box (Sample Size Box), asample to chunk box (Sample To Chunk Box), a chunk offset box (ChunkOffset Box), and a subsample information box (Subsample InformationBox).

The sample description box stores information on codec, image size andthe like. For example, information such as encoding parameters is storedin an HEVC sample entry within the sample description box.

The time to sample box stores information on time of a sample. Thesample size box stores information on a size of a sample. The sample tochunk box stores information on a position of data in a sample. Thechunk offset box stores information on offset of data. The subsampleinformation box stores information on a subsample.

<Division Method>

Tiles may be equal divisions of an entire image as illustrated in anexample in A of FIG. 10, or may be unequal divisions of an entire imageas illustrated in an example in B of FIG. 10. In other words, imagesizes of respective tile images constituting an entire image may beeither a uniform size, or different sizes.

<Application>

Examples of an application having this tile structure may include anapplication which controls a size of a partial image to be displayed.

It is assumed that an entire image 10 illustrated in A of FIG. 10 istiled and divided into a plurality of tile images 11 having an identicalsize. For displaying this image on a mobile device 21 having a smalldisplay size, for example, an application displays partial images 12constituted by four (2×2) tile images. On the other hand, for displayingthis image on a television signal receiver (TV) 22 having a largedisplay size, for example, an application displays partial images 13constituted by thirty (6×5) tile images. It is therefore considered touse such an application which controls an image size of a partial imageto be displayed in accordance with performance or the like of a terminalwhich displays images.

In case of an example illustrated in B of FIG. 10, image sizes ofrespective tile images are not equalized. An application displays animage of a tile 3 (Tile3) to display an image having HD resolution,displays images of a tile 2 (Tile2) through a tile 4 (Tile4) to displayimages having cinema resolution, and displays images of a tile 1 (Tile1)through a tile 5 (Tile5) to display larger images in an extended size(EXT). It is therefore considered to use such an application whichcontrols resolution or an aspect ratio of a display image by controllingan image size of a partial image to be displayed.

This adaptive control over a size of a partial image to be distributed(control over the number of tile images to be distributed) in accordancewith an image size of a partial image to be displayed eliminates thenecessity of distribution of a not-displayed unnecessary part of theimage to these applications. This control therefore adaptively controlsloads on a server, a terminal, a transmission path and the like, andreduces a rise of unnecessary loads.

<Adaptive Provision of Tile Image>

For adaptively providing data on a partial image, i.e., selecting anarbitrary partial image and providing data on the selected partial imageby utilizing tile structure of images, for example, partial imageinformation corresponding to information on a partial image constitutinga part of an entire image is inserted into MPD, for example.

A partial image to be provided may be an arbitrary image as long as apart of an entire image constitutes the partial image. A shape, a sizeand the like of the partial image may be arbitrarily determined. Forexample, a partial image may be an arbitrary part as long as the part isencodable independently from other parts. It is assumed hereinafter thata partial image is an image in units of tile as described above forconvenience of explanation. More specifically, it is assumed that apartial image is constituted by one or a plurality of tile images.

For example, description for a tile is defined as illustrated in A ofFIG. 11 while utilizing a descriptor type element (DescriptorTypeelement) in MPD. In addition, a view type (viewtype) indicating a typeof an image shown by a corresponding element is defined as illustratedin B of FIG. 11, for example.

<Adaptive Provision of Tile Images in Plural Layers>

Incidentally, examples of an image encoding and decoding system includea layer encoding and layer decoding system which efficiently encodes animage constituted by a plurality of layers by using prediction betweenlayers or other methods.

In this layer encoding and layer decoding system for image distributionor other purposes, adaptive provision of a partial image in each of aplurality of layers is needed. According to conventional MPEG-DASHstandards, however, adaptive provision of partial image data in aplurality of layers is difficult.

For overcoming this problem, there is generated a file which storesinformation on a whole of a base layer of encoded data generated bylayer encoding of image data, information on respective partial areas ofan image of the base layer, information on a whole of an enhancementlayer of encoded data, and information on respective partial areas of animage of the enhancement layer such that each of the information isstored in corresponding one of tracks different from each other.

It is also allowed to store, in the track storing the information on thewhole of the base layer, encoding information indicating an encodingsystem of the baser layer, to store, in each of the tracks storing theinformation on the partial areas of the base layer, encoding informationindicating that the corresponding track stores only the information onthe corresponding partial area of the base layer; to store, in the trackstoring the information on the whole of the enhancement layer, encodinginformation indicating an encoding system of the enhancement layer, andto store, in each of the tracks storing the information on the partialareas of the enhancement layer, encoding information indicating that thecorresponding track stores only information on the corresponding partialarea of the enhancement layer.

It is further allowed to store, in the track storing the information onthe whole of the base layer, information indicating reference to therespective tracks storing the information on the partial areas of thebase layer, and information indicating a type of the reference, tostore, in each of the tracks storing the information on the partialareas of the base layer, information indicating reference to the trackstoring the information on the whole of the base layer, and informationindicating a type of the reference, to store, in the track storinginformation on the whole of the enhancement layer, informationindicating reference to the track storing the information on the wholeof the base layer and information indicating a type of the reference,and information indicating reference to the tracks storing theinformation on the partial areas of the enhancement layer andinformation indicating a type of the reference, and to store, in each ofthe tracks storing the information on the partial areas of theenhancement layer, information indicating reference to the track storingthe information on the whole of the enhancement layer, and informationindicating a type of the reference.

This configuration allows adaptive provision of a partial image of animage in each layer of an image constituted by a plurality of layers.

In addition, there is generated metadata which contains information on awhole of a base layer of encoded data generated by layer encoding ofimage data, information on each of partial areas of an image of the baselayer, information on a whole of an enhancement layer of the encodeddata, and information on each of partial areas of an image of theenhancement layer.

Note as follows: the information storing information on the whole of thebase layer may contain encoding information indicating an encodingsystem of the base layer; the information on each of partial areas ofthe base layer may contain encoding information indicating that onlyinformation on the corresponding partial area of the base layer iscontained; the information storing the whole of the enhancement layermay contain encoding information indicating an encoding system of theenhancement layer; and the information on each of partial areas of theenhancement layer may contain encoding information indicating that onlyinformation on the corresponding partial area of the enhancement layeris contained.

In addition, the information on the whole of the base layer may containinformation indicating reference to the information on the partial areasof the base layer and information indicating a type of the reference.The information on each of the partial areas of the base layer maycontain information indicating reference to the information on the wholeof the base layer and information on a type of the reference. Theinformation on the whole of the enhancement layer may containinformation indicating reference to the information on the whole of thebase layer and information indicating a type of the reference, andinformation indicating reference to the information on the partial areasof the enhancement layer and information indicating a type of thereference. The information on the partial areas of the enhancement layermay contain information indicating reference to the information on thewhole of the enhancement layer and information on a type of thereference.

This configuration allows adaptive provision of a partial image of animage in each layer of an image constituted by a plurality of layers.

<Use Case>

Described herein is an example of the present technology which performslayer encoding of double-layered image data constituted by a base layerand an enhancement layer by utilizing prediction between layers, anddistributes the encoded image data. Needless to say, the number oflayers of image data may be arbitrarily determined, such as three layersor more. In addition, it is assumed that an image of a base layer has ahigh correlation with an image of an enhancement layer (both images arebasically identical images except for predetermined image parameterssuch as resolution), and has resolution lower than resolution of theimage of the enhancement layer.

It is further assumed that an image of each layer is divided into fourtiles (partial areas) in total (two in vertical direction and two inhorizontal direction) for convenience of explanation. Needless to say,the number of divided tiles in an image of each layer may be arbitrarilydetermined.

An example of use of this layer encoding is initially described. Forenlarged display of an image of a base layer (BL) having low resolution(low image quality) as in an example illustrated in FIG. 12, forexample, partial tiles are extracted and enlarged to an original imagesize. In this case, an enlargement rate of display of partial tilesextracted from an image of an enhancement layer (EL) having highresolution (high image quality) may become lower than the foregoingenlargement rate of the base layer. According to the example illustratedin FIG. 12, the necessity of enlargement is eliminated when the imagesof the enhancement layer are used. Accordingly, while contents of imagesto be displayed are substantially equivalent in both the layers, imagequality of the enhancement layer becomes higher in display.

In this situation, deterioration of image quality decreases (imagequality of provided images improves) when images to be distributed areswitched from the base layer to the enhancement layer.

According to conventional MPEG-DASH, prepared streams to be distributedare switchable between low-resolution streams and high-resolutionstreams in accordance with situations.

According to this data distribution, however, data is generallydistributed sufficiently earlier than display timing such that data forseveral seconds to several tens of seconds, for example, is retained ina buffer on a receiving side for stabilization of image reproductioneven in streaming distribution, for example.

Accordingly, when a stream to be distributed in an image at a time T1 isswitched (switched from stream 1 to stream 2) in a manner describedabove as illustrated in A of FIG. 13, for example, all data in thestream 1 accumulated in the buffer from the time T1 to a time T2 isdiscarded. Thereafter, distribution of data in the stream 2 starts fromthe time T2.

Switching in an opposite direction (from stream 2 to stream 1) issimilarly performed. When a stream to be distributed in an image at atime T3 is switched from the stream 2 to the stream 1, for example, allof data in the stream 2 accumulated in the buffer from the time T3 to atime T4 is discarded. Thereafter, distribution of data in the stream 1starts from the time T3.

Accordingly, data in the buffer is discarded and wasted every time astream to be distributed is switched. Moreover, data remaining in thebuffer becomes zero, in which condition of reproduction may becomeunstable.

According to layer-encoded data, however, a stream of a base layer isconstantly distributed as illustrated in B of FIG. 13. A stream of anenhancement layer is distributed only when an image of the enhancementlayer is displayed.

Accordingly, the necessity of discarding data in the buffer iseliminated at the time of switching of an image for display from a baselayer to an enhancement layer. In other words, encoded data of the baselayer accumulated in the buffer is continuously used after switching.Moreover, prediction between layers has been performed for a start ofdistribution of a stream of an enhancement layer. In this case, a datavolume of the stream of the enhancement layer becomes smaller than adata volume of the stream 2 in A of FIG. 13. In this case, data in thebuffer does not become zero. Accordingly, image display is morestabilized than in the case of A of FIG. 13 even after the time T1 ofswitching.

This is also applicable to switching in an opposite direction (fromenhancement layer to base layer). When an image to be displayed isswitched from an enhancement layer to a base layer at the time T3, forexample, all of data on the enhancement layer accumulated in the bufferfrom the time T3 to the time T4 is discarded. However, a volume of datato be discarded is smaller than a volume of discarded data in the stream2 in A of FIG. 13. In addition, data in the base layer has beendistributed before the time T3, wherefore data in the buffer does notbecome zero. Accordingly, image display is more stabilized than in thecase of A of FIG. 13 even after the time T3 of switching.

The following effects are offered, for example, by realizing adaptiveprovision of a partial image of an image in each layer of an imageconstituted by a plurality of layers. The functions of the followingeffects are realized in a distribution format, such as an MP4 file andMPD, according to the present technology.

<File Generation Device>

A device which generates MP4 files and MPD described above ishereinafter described. FIG. 14 is a block diagram illustrating a generalconfiguration example of a file generation device corresponding to aninformation processing device according to an embodiment to which thepresent technology has been applied. A file generation device 100 inFIG. 14 is a device which generates MP4 files for storing encoded datain respective layers obtained by layer encoding of an image of a baselayer (base layer (BL) image) and an image of a enhancement layer(enhancement layer (EL) image), and generates MPD for controllingdistribution of MP4 files thus generated.

As illustrated in FIG. 3, the file generation device 100 includes a baselayer encoding unit 101, an enhancement layer encoding unit 102, an MP4file generation unit 103, and an MPD generation unit 104.

The base layer encoding unit 101 encodes an input base layer image (BLimage) by a predetermined encoding system (such as HEVC) to generatebase layer encoded data. The base layer encoding unit 101 suppliesgenerated base layer encoded data (HEVC) to the MP4 file generation unit103. The base layer encoding unit 101 further supplies a base layerimage (such as decoded image) to the enhancement layer encoding unit 102as a reference image. The base layer encoding unit 101 further suppliesencoding information to the enhancement layer encoding unit 102 asinformation about encoding.

The enhancement layer encoding unit 102 encodes an input enhancementlayer image (EL image) by a predetermined encoding system (such as SHVC)to generate enhancement layer encoded data. In this case, theenhancement layer encoding unit 102 performs prediction between layerson the basis of a reference image and encoding information supplied fromthe base layer encoding unit 101. This prediction reduces a drop ofencoding efficient. The enhancement layer encoding unit 102 suppliesgenerated enhancement layer encoded data (SHVC) to the MP4 filegeneration unit 103.

The MP4 file generation unit 103 generates a file (MP4 file) whichstores base layer encoded data (HEVC) supplied from the base layerencoding unit 101, and enhancement layer encoded data (SHVC) suppliedfrom the enhancement layer encoding unit 102. The MP4 file generationunit 103 stores the base layer encoded data (HEVC) and the enhancementlayer encoded data (SHVC) in different tracks of MP4 files. The MP4 filegeneration unit 103 outputs generated MP4 files. The MP4 file generationunit 103 further supplies generated MP4 files to the MPD generation unit104.

The MPD generation unit 104 generates MPD which controls distribution ofMP4 files supplied from the MP4 file generation unit 103. The MPDgeneration unit 104 outputs the generated MP4 files.

<Configuration Example of MP4 File>

The MP4 file generation unit 103 of the file generation device 100generates MP4 files illustrated in FIGS. 15 and 16, for example, as MP4files storing base layer encoded data (HEVC) and enhancement layerencoded data (SHVC).

As illustrated in FIGS. 15 and 16, the MP4 file generation unit 103generates tracks for storing information on a whole of a base layer,information on respective tiles of a base layer image, information on awhole of an enhancement layer, and information on respective tiles of anenhancement layer image, respectively. In this case, the MP4 filegeneration unit 103 files the respective tracks as MP4 files.

More specifically, the MP4 file generation unit 103 generates, for thebase layer, an MP4 file which stores a track 1 (Track1) storinginformation on the whole of the base layer, an MP4 file which stores atrack 2 (Track2) storing information on a tile 1 (tile1) of the baselayer image, an MP4 file which stores a track 3 (Track3) storinginformation on a tile 2 (tile2) of the base layer image, an MP4 filewhich stores a track 4 (Track4) storing information on a tile 3 (tile3)of the base layer image, an MP4 file which stores a track 5 (Track5)storing information on a tile 4 (tile4) of the base layer image (FIG.15).

Similarly, the MP4 file generation unit 103 generates, for theenhancement layer, an MP4 file which stores a track 6 (Track6) storinginformation on the whole of the enhancement layer, an MP4 file whichstores a track 7 (Track7) storing information on a tile 1 (tile1) of theenhancement layer image, an MP4 file which stores a track 8 (Track5)storing information on a tile 2 (tile2) of the enhancement layer image,an MP4 file which stores a track 9 (Track9) storing information on atile 3 (tile3) of the enhancement layer image, an MP4 file which storesa track 10 (Track10) storing information on a tile 4 (tile4) of theenhancement layer image (FIG. 16).

The track 1 (Track1) storing the information on the whole of the baselayer stores an extractor sample (EXT sample) corresponding to referenceinformation indicating reference to each tile of the base layer. Inaddition, a sample entry of the track 1 contains encoding informationindicating HEVC as the encoding system of the base layer (SampleEntry=‘hvc2’). Moreover, this sample entry contains an hvcC box (hvcCbox) storing configuration information necessary for decoding of HEVCencoded data. Furthermore, a track reference (Track Reference)corresponding to information about reference between the track 1 andother tracks contains indication of the presence of information on thetrack storing a sample for reference, and information on a type of thereference (scal=2, 3, 4, 5). In other words, the track referencecontains information indicating that the track 1 refers to the tracks 2through 5 by a reference type of “scal”.

Each of the track 2 (Track2) through the track 5 (Track5) storinginformation on the tiles of the base layer image stores a sample of thecorresponding tile of the base layer (tile1 sample, tile2 sample, tile3sample, or tile4 sample). In addition, each sample entry of the tracks 2through 5 contains encoding information indicating that the track storesonly a slice of the tile of the base layer (Sample Entry=‘hvt1’). Inaddition, each of the sample entries contains an hvtC box (hvtC box)which stores configuration information necessary for decoding of data inthe track. Moreover, each track reference (Track Reference) of thesetracks indicates reference to the track 1 by a reference type of “tbas”(tbas=1). Furthermore, each of these tracks contains, as a tile regiongroup entry (TileRegionGroupEntry), identification information foridentifying the corresponding tile of the base layer, a horizontalposition of the corresponding tile (offset), a vertical position of thecorresponding tile (offset), a horizontal size of the corresponding tile(width), a vertical size of the corresponding tile (height), and others.

The track 6 (Track6) storing the information on the whole of theenhancement layer stores an extractor sample (EXT sample) correspondingto reference information indicating reference to each tile of theenhancement layer. In addition, a sample entry of the track 6 containsencoding information indicating SHVC as the encoding system of theenhancement layer (Sample Entry=‘lhv1’). Moreover, this sample entrycontains an hvcC box (hvcC box) storing configuration informationnecessary for decoding of SHVC encoded data.

Furthermore, a track reference (Track Reference) of the track 6indicates reference to the tracks 7 through 10 by a reference type of“scal” (scal=7, 8, 9, 10). The track reference (Track Reference) of thetrack 6 indicates reference to the track 1 storing information on thewhole of the base layer by a reference type of “sbas” (sbas=1). Thisconfiguration allows reference from the tracks of the enhancement layersto the tracks of the base layer.

Each of the track 7 (Track7) through the track 10 (Track10) storinginformation on the tiles of the enhancement layer image stores a sampleof the corresponding tile of the enhancement layer (tile1 sample, tile2sample, tile3 sample, or tile4 sample).

In addition, each sample entry of the tracks 7 through 10 containsencoding information indicating that the track stores only a slice ofthe tile of the enhancement layer (Sample Entry=‘lht1’). In addition,each of the sample entries contains an lhtC box (lhtC box) which storesconfiguration information necessary for decoding of data in the track.As described above, the encoding information on the respective tracksstoring the tiles of the enhancement layer is different from theencoding information on the respective tracks storing tiles of the baselayer. Accordingly, the tracks storing the tiles of the enhancementlayer are identifiable as information storing the tiles of theenhancement layer.

Moreover, each track reference (Track Reference) of these tracksindicates reference to the track 6 by a reference type of “tbas”(tbas=6). Accordingly, each of the track references indicates that eachof the tracks storing the tiles of the enhancement layers refers not tothe track 1 storing information on the whole of the base layer, but tothe track 6 storing information on the whole of the enhancement layer.

Furthermore, each of these tracks contains, as a tile region group entry(TileRegionGroupEntry), identification information for identifying thecorresponding tile of the enhancement layer, a horizontal position ofthe corresponding tile (offset), a vertical position of thecorresponding tile (offset), a horizontal size of the corresponding tile(width), a vertical size of the corresponding tile (height), and others.

The MP4 file generation unit 103 which generates MP4 files having thisconfiguration allows control of distribution of each tile in theenhancement layer as well as in the base layer. Accordingly, a partialimage (such as tile) of an image in each layer of an image constitutedby a plurality of layers is adaptively provided in image distribution.

<Flow of MP4 File Generation Process>

Described hereinafter with reference to a flowchart shown in FIG. 17 isan example of a flow of an MP4 file generation process corresponding toa process for generating MP4 files described above, and executed by thefile generation device 100 illustrated in FIG. 14.

With a start of the MP4 file generation process, the base layer encodingunit 101 encodes a base layer image in step S101.

In step S102, the MP4 file generation unit 103 generates a track of anextractor of the base layer, and files the track as an MP4 file.

In step S103, the MP4 file generation unit 103 generates a sample entryin the track of the extractor of the base layer.

In step S104, the MP4 file generation unit 103 generates a track of eachtile of the base layer, and files the generated tracks as MP4 files.

In step S105, the MP4 file generation unit 103 generates a tile regiongroup entry in the track of each tile of the base layer.

In step S106, the MP4 file generation unit 103 generates a sample entryin the track of each tile of the base layer.

In step S107, the MP4 file generation unit 103 generates a trackreference indicating the tracks of the respective tiles of the baselayer in the track of the extractor of the base layer.

In step S108, the MP4 file generation unit 103 generates a trackreference indicating the track of the extractor of the base layer in thetrack of each tile of the base layer.

In step S109, the enhancement layer encoding unit 102 performsprediction between layers with reference to a reference image of thebase layer as necessary, and encodes an enhancement layer image.

In step S110, the MP4 file generation unit 103 generates a track of anextractor of the enhancement layer, and files the track as an MP4 file.

In step S111, the MP4 file generation unit 103 generates a sample entryin the track of the extractor of the enhancement layer.

In step S112, the MP4 file generation unit 103 generates a track of eachtile of the enhancement layer, and files the generated tracks as MP4files.

In step S113, the MP4 file generation unit 103 generates a tile regiongroup entry in the track of each tile of the enhancement layer.

In step S114, the MP4 file generation unit 103 generates a sample entryin the track of each tile of the enhancement layer.

In step S115, the MP4 file generation unit 103 generates a trackreference indicating the tracks of the respective tiles of theenhancement layer in the track of the extractor of the enhancementlayer.

In step S116, the MP4 file generation unit 103 generates a trackreference indicating the track of the extractor of the enhancement layerin the track of each tile of the enhancement layer.

In step S117, the MP4 file generation unit 103 generates a trackreference indicating the track of the extractor of the base layer in thetrack of the extractor of the enhancement layer.

In step S118, the MP4 file generation unit 103 outputs the respectiveMP4 files thus generated.

As described above, the file generation device 100 generates MP4 fileshaving the configuration illustrated in FIGS. 15 and 16, and allowscontrol of distribution of each tile in the enhancement layer as well asin the base layer by performing the MP4 file generation processdescribed above. Accordingly, a partial image (such as tile) of an imagein each layer of an image constituted by a plurality of layers isadaptively provided in image distribution.

<Configuration Example of MPD>

The MPD generation unit 104 of the file generation device 100 generatesMPD having a configuration illustrated in FIGS. 18 and 19 from MP4 filesgenerated by the MP4 file generation unit 103 in the manner describedabove, for example.

As illustrated in FIGS. 18 and 19, the MPD generation unit 104 generatesmetadata containing information on a whole of a base layer, informationon respective tiles of a base layer image, information on a whole of anenhancement layer, and information on respective tiles of an enhancementlayer image. In a state that each track has been filed as an MP4 file,the MPD generation unit 104 sets an adaptation set (AdaptationSet) foreach MP4 file (track), and sets (registers) an MP4 file for each segment(Segment).

More specifically, the MPD generation unit 104 sets an adaptation setstoring information on the whole of the base layer, sets arepresentation (Representation) in an order lower than the adaptationset, sets a segment (Segment) in an order lower than the representation,and registers an MP4 file (Bl_bs.mp4) storing information on the wholeof the base layer in the segment.

The MPD generation unit 104 sets encoding information belonging to asample entry of the MP4 file in the adaptation set (<codecs=“hvc2”>).The MPD generation unit 104 further sets identification information inthe representation to indicate that stored information is information onthe whole of the base layer (<id=“bl_bs”>). The MPD generation unit 104further sets information and the like contained in the track referenceof the MP4 file as association information indicating a referencerelationship between adaptation sets and a type of reference. Forexample, the following information is set.

<associationid=“bl_tile1,bl_tile2,bl_tile3,bl_tile4”associationType=“scal”>

Similarly, the MPD generation unit 104 sets an adaptation set storinginformation on the tile 1 (tile1) of the base layer image, sets arepresentation in an order lower than the adaptation set, sets a segmentin an order lower than the representation, and registers the MP4 file(Bl_tile1.mp4) storing information on the tile 1 of the base layer imagein the segment.

The MPD generation unit 104 sets encoding information belonging to thesample entry of the MP4 file in the adaptation set (<codecs=“hvt1”>).The MPD generation unit 104 further sets an essential property(EssentialProperty) in the adaptation set, sets a tile scheme(schemeIdUri=“urn:mpeg:dash:srd:2013”) as an element of the essentialproperty, and sets information on a tile region group entry of the MP4file and the like as a value (value). For example, the followinginformation is set.

<EssentialProperty schemeIdUri=“urn:mpeg:dash:srd:2013”value=“1, 0, 0,960, 540, 1920, 1080”>

The MPD generation unit 104 further sets identification information inthe representation to indicate that stored information is information onthe tile 1 of the base layer (<id=“bl_tile1”>). The MPD generation unit104 further sets information and the like contained in the trackreference of the MP4 file as association information indicating areference relationship between adaptation sets and a type of reference.For example, the following information is set.

<associationid=“bl_bs”associationType=“tbas”>

Similarly, the MPD generation unit 104 sets an adaptation set storinginformation on the tile 2 (tile2) of the base layer image, sets arepresentation in an order lower than the adaptation set, sets a segmentin an order lower than the representation, and registers the MP4file(Bl_tile2.mp4) storing information on the tile 2 of the base layerimage in the segment.

The MPD generation unit 104 sets encoding information belonging to thesample entry of the MP4 file in the adaptation set (<codecs=“hvt1”>).The MPD generation unit 104 further sets an essential property(EssentialProperty) in the adaptation set, sets a tile scheme(schemeIdUri=“urn:mpeg:dash:srd:2013”) as an element of the essentialproperty, and sets information on a tile region group entry of the MP4file and the like as a value (value). For example, the followinginformation is set.

<EssentialPropertyschemeIdUri=“urn:mpeg:dash:srd:2013”value=“1,960,0,960,54 0,1920,1080”>

The MPD generation unit 104 further sets identification information inthe representation to indicate that stored information is information onthe tile 2 of the base layer (<id=“bl_tile2”>). The MPD generation unit104 further sets information and the like contained in the trackreference of the MP4 file as association information indicating areference relationship between adaptation sets and a type of reference.For example, the following information is set.

<associationid=“bl_bs”associationType=“tbas”>

Similarly, the MPD generation unit 104 sets an adaptation set storinginformation on the tile 3 (tile3) of the base layer image, sets arepresentation in an order lower than the adaptation set, sets a segmentin an order lower than the representation, and registers the MP4 file(Bl_tile3.mp4) storing information on the tile 3 of the base layer imagein the segment.

The MPD generation unit 104 sets encoding information belonging to thesample entry of the MP4 file in the adaptation set (<codecs=“hvt1”>).The MPD generation unit 104 further sets an essential property(EssentiaiProperty) in the adaptation set, sets a tile scheme(schemeIdUri=“urn:mpeg:dash:srd:2013”) as an element of the essentialproperty, and sets information on a tile region group entry of the MP4file and the like as a value (value). For example, the followinginformation is set.

<EssentialPropertyschemeIdUri=“urn:mpeg:dash:srd:2013”value=“1,0,540,960,54 0,1920,1080”>

The MPD generation unit 104 further sets identification information inthe representation to indicate that stored information is information onthe tile 3 of the base layer (<id=“bl_tile3”>). The MPD generation unit104 further sets information and the like contained in the trackreference of the MP4 file as association information indicating areference relationship between adaptation sets and a type of reference.For example, the following information is set.

<associationid=“bl_bs”associationType=“tbas”>

Similarly, the MPD generation unit 104 sets an adaptation set storinginformation on the tile 4 (tile4) of the base layer image, sets arepresentation in an order lower than the adaptation set, sets a segmentin an order lower than the representation, and registers the MP4 file(Bl_tile4.mp4) storing information on the tile 4 of the base layer imagein the segment.

The MPD generation unit 104 sets encoding information belonging to thesample entry of the MP4 file in the adaptation set (<codecs:=“hvt1”>).The MPD generation unit 104 further sets an essential property(EssentialProperty) in the adaptation set, sets a tile scheme(schemeIdUri=“urn:mpeg:dash:srd:2013”) as an element of the essentialproperty, and sets information on a tile region group entry of the MP4file and the like as a value (value). For example, the followinginformation is set.

<EssentialPropertyschemeIdUri=“urn:mpeg:dash:srd:2013”value=“1,960,540,960,540,1920,1080”>

The MPD generation unit 104 further sets identification information inthe representation to indicate that stored information is information onthe tile 4 of the base layer (<id=“bl_tile4”>). The MPD generation unit104 further sets information and the like contained in the trackreference of the MP4 file as association information indicating areference relationship between adaptation sets and a type of reference.For example, the following information is set.

<associationid=“bl_bs”associationType=“tbas”>

FIG. 18 illustrates the foregoing configuration.

In addition, the MPD generation unit 104 sets an adaptation set storinginformation on the whole of the enhancement layer, sets a representationin an order lower than the adaptation set, sets a segment in an orderlower than the representation, and registers an MP4 file(El_bs.mp4)storing information on the whole of the enhancement layer in thesegment.

The MPD generation unit 104 sets encoding information belonging to asample entry of the MP4 file in the adaptation set (<codecs=“lhv1”>).The MPD generation unit 104 further sets identification information inthe representation to indicate that stored information is information onthe whole of the base layer (<id=“el_bs”>). The MPD generation unit 104further sets information and the like contained in the track referenceof the MP4 file as association information indicating a referencerelationship between adaptation sets and a type of reference. Forexample, the following information is set.

<associationid=“bl_bs”associationType=“sbas”><associationid=“el_tile1,el_tile2,el_tile3,el_tile4”associationType=“scal”>

This configuration allows reference to the adaptation set of the baselayer from the adaptation set of the enhancement layer.

Similarly, the MPD generation unit 104 sets an adaptation set storinginformation on the tile 1 (tile1) of the enhancement layer image, sets arepresentation in an order lower than the adaptation set, sets a segmentin an order lower than the representation, and registers the MP4 file(el_tile1.mp4) storing information on the tile 1 of the enhancementlayer image in the segment.

The MPD generation unit 104 sets encoding information belonging to thesample entry of the MP4 file in the adaptation set (<codecs=“lht1”>).The MPD generation unit 104 further sets an essential property(EssentialProperty) in the adaptation set, sets a tile scheme(schemeIdUri=“urn:mpeg:dash:srd:2013”) as an element of the essentialproperty, and sets information on a tile region group entry of the MP4file and the like as a value (value). For example, the followinginformation is set.

<EssentialPropertyschemeIdUri=“urn:mpeg:dash:srd:2013”value=“1,0,0,1920,108 0,3840,2160”>

The MPD generation unit 104 further sets identification information inthe representation to indicate that stored information is information onthe tile 1 of the enhancement layer (<id=“el_tile1”>). The MPDgeneration unit 104 further sets information and the like contained inthe track reference of the MP4 file as association informationindicating a reference relationship between adaptation sets and a typeof reference. For example, the following information is see.

<associationid=“el_bs”associationType=“tbas”>

Similarly, the MPD generation unit 104 sets an adaptation set storinginformation on the tile 2 (tile2) of the enhancement layer image, sets arepresentation in an order lower than the adaptation set, sets a segmentin an order lower than the representation, and registers an MP4 file(el_tile2.mp4) storing information on the tile 2 of the enhancementlayer image in the segment.

The MPD generation unit 104 sets encoding information belonging to thesample entry of the MP4 file in the adaptation set (<codecs=“lht1”>).The MPD generation unit 104 further sets an essential property(EssentialProperty) in the adaptation set, sets a tile scheme(schemeIdUri=“urn:mpeg:dash:srd:2013”) as an element of the essentialproperty, and sets information on a tile region group entry of the MP4file and the like as a value (value). For example, the followinginformation is set.

<EssentialPropertyschemeIdUri=“urn:mpeg:dash:srd:2013”value=“1,1920,0,1920,1080,3840,2160”>

The MPD generation unit 104 further sets identification information inthe representation to indicate that stored information is information onthe tile 2 of the enhancement layer (<id=“el_tile2”>). The MPDgeneration unit 104 further sets information and the like contained inthe track reference of the MP4 file as association informationindicating a reference relationship between adaptation sets and a typeof reference. For example, the following information is set.

<associationid=“el_bs”associationType=“tbas”>

Similarly, the MPD generation unit 104 sets an adaptation set storinginformation on the tile 3 (tile3) of the enhancement layer image, sets arepresentation in an order lower than the adaptation set, sets a segmentin an order lower than the representation, and registers an MP4 file(el_tile3.mp4) storing information on the tile 3 of the enhancementlayer image in the segment.

The MPD generation unit 104 sets encoding information belonging to thesample entry of the MP4 file in the adaptation set (<codecs=“lht1”>).The MPD generation unit 104 further sets an essential property(EssentialProperty) in the adaptation set, sets a tile scheme(schemeIdUri=“urn:mpeg:dash:srd:2013”) as an element of the essentialproperty, and sets information on a tile region group entry of the MP4file and the like as a value (value). For example, the followinginformation is set.

<EssentialPropertyschemeIdUri=“urn:mpeg:dash:srd:2013”value=“1,0,1080,1920,1080,3840,2160”>

The MPD generation unit 104 further sets identification information inthe representation to indicate that stored information is information onthe tile 3 of the enhancement layer (<id=“el_tile3”>). The MPDgeneration unit 104 further sets information and the like contained inthe track reference of the MP4 file as association informationindicating a reference relationship between adaptation sets and a typeof reference. For example, the following information is set.

<associationid=“el_bs”associationType=“tbas”>

Similarly, the MPD generation unit 104 sets an adaptation set storinginformation on the tile 4 (tile4) of the enhancement layer image, sets arepresentation in an order lower than the adaptation set, sets a segmentin an order lower than the representation, and registers an MP4 file(el_tile4.mp4) storing information on the tile 4 of the enhancementlayer image in the segment.

The MPD generation unit 104 sets encoding information belonging to thesample entry of the MP4 file in the adaptation set (<codecs=“lht1”>).The MPD generation unit 104 further sets an essential property(EssentialProperty) in the adaptation set, sets a tile scheme(schemeIdUri=“urn:mpeg:dash:srd:2013”) as an element of the essentialproperty, and sets information on a tile region group entry of the MP4file and the like as a value (value). For example, the followinginformation is set.

<EssentialPropertyschemeIdUri=“urn:mpeg:dash:srd:2013”value=“1,1920,1080,1920,1080,3840,2160”>

The MPD generation unit 104 further sets identification information inthe representation to indicate that stored information is information onthe tile 4 of the enhancement layer (<id=“el_tile2”>). The MPDgeneration unit 104 further sets information and the like contained inthe track reference of the MP4 file as association informationindicating a reference relationship between adaptation sets and a typeof reference. For example, the following information is set.

<associationid=“el_bs”associationType=“tbas”>

FIG. 18 illustrates the foregoing configuration.

As described above, the encoding information on the adaptation sets ofthe respective tiles of the enhancement layer is different from theencoding information on the adaptation sets of the respective tiles ofthe base layer. Accordingly, the adaptation sets of the tiles of theenhancement layer are identifiable as adaptation sets of the tiles ofthe enhancement layer.

In addition, the association information on the representationsbelonging to these adaptation sets indicates reference to “el_bs”, and atype of reference “tbas”. In this case, reference not to the adaptationset (bl_bs) of the information on the whole of the base layer, but toadaptation set (el_bs) of the information on the whole of theenhancement layer is allowed from the adaptation sets of the respectivetiles of the enhancement layer.

The MPD generation unit 104 which generates MPD having thisconfiguration allows control of distribution for each tile in theenhancement layer as well as in the base layer. Accordingly, a partialimage (such as tile) of an image in each layer of an image constitutedby a plurality of layers is adaptively provided in image distribution.

<Flow of MPD Generation Process>

Described with reference to a flowchart shown in FIGS. 20 and 21 is anexample of a flow of an MPD generation process performed by the filegeneration device 100 illustrated in FIG. 14 and corresponding to aprocess for generating MPD described above.

With a start of the MPD generation process, the base layer encoding unit101 encodes a base layer image in step S121 in FIG. 20.

In step S122, the enhancement layer encoding unit 102 performsprediction between layers with reference to a reference image of thebase layer as necessary, and encodes an enhancement layer image.

In step S123, the MP4 file generation unit 103 generates an MP4 filestoring encoded data on the base layer and encoded data on theenhancement layer. This processing is similar to the correspondingprocessing in the MP4 file generation process described with referenceto the flowchart in FIG. 17. Accordingly, the same explanation of thisprocessing is not repeated.

In step S124, the MPD generation unit 104 generates an adaptation set ofan extractor of the base layer.

In step S125, the MPD generation unit 104 sets encoding information inthe adaptation set of the extractor of the base layer.

In step S126, the MPD generation unit 104 generates a representation ofthe extractor of the base layer.

In step S127, the MPD generation unit 104 sets identificationinformation in the representation of the extractor of the base layer.

In step S128, the MPD generation unit 104 sets association informationin the representation of the extractor of the base layer to indicateassociation with the respective tiles of the base layer.

In step S129, the MPD generation unit 104 generates a segment of theextractor of the base layer.

In step S130, the MPD generation unit 104 generates an adaptation set ofeach tile of the base layer.

In step S131, the MPD generation unit 104 sets encoding information inthe adaptation set of each tile of the base layer.

In step S132, the MPD generation unit 104 sets an essential property inthe adaptation set of each tile of the base layer.

In step S133, the MPD generation unit 104 generates a representation ofeach tile of the base layer.

In step S134, the MPD generation unit 104 sets identificationinformation in the representation of each tile of the base layer.

In step S135, the MPD generation unit 104 sets association informationin the representation of each tile of the base layer to indicateassociation with the extractor of the base layer.

In step S136, the MPD generation unit 104 generates a segment of eachtile of the base layer.

In step S141 in FIG. 21, the MPD generation unit 104 generates anadaptation set of an extractor of the enhancement layer.

In step S142, the MPD generation unit 104 sets encoding information inthe adaptation set of the extractor of the enhancement layer.

In step S143, the MPD generation unit 104 sets association informationin the adaptation set of the extractor of the enhancement layer toindicate association with the extractor of the base layer.

In step S144, the MPD generation unit 104 generates a representation ofthe extractor of the enhancement layer.

In step S145, the MPD generation unit 104 sets identificationinformation in the representation of the extractor of the enhancementlayer.

In step S146, the MPD generation unit 104 sets association informationin the representation of the extractor of the enhancement layer toindicate association with the respective tiles of the enhancement layer,and association information in the representation of the extractor ofthe enhancement layer to indicate association with the extractor of thebase layer.

In step S147, the MPD generation unit 104 generates a segment of theextractor of the enhancement layer.

In step S148, the MPD generation unit 104 generates the adaptation setof each tile of the enhancement layer.

In step S149, the MPD generation unit 104 sets encoding information inthe adaptation set of each tile of the enhancement layer.

In step S150, the MPD generation unit 104 sets an essential property inthe adaptation set of each tile of the enhancement layer.

In step S151, the MPD generation unit 104 generates a representation ofeach tile of the enhancement layer.

In step S152, the MPD generation unit 104 sets identificationinformation in the representation of each tile of the enhancement layer.

In step S153, the MPD generation unit 104 sets association informationin the representation of each tile of the enhancement layer to indicateassociation with the extractor of the base layer.

In step S154, the MPD generation unit 104 generates a segment of eachtile of the enhancement layer.

In step S155, the MPD generation unit 104 outputs MPD thus generated.

The file generation device 100 generates MPD having the configurationillustrated in FIGS. 18 and 19, and allows control of distribution foreach tile in the enhancement layer as well as in the base layer byperforming the MPD generation process described above. Accordingly, apartial image (such as tile) of an image in each layer of an imageconstituted by a plurality of layers is adaptively provided in imagedistribution.

<File Reproduction Device>

A device for reproducing MP4 files and MPD thus generated is hereinafterdescribed. FIG. 22 is a block diagram illustrating a generalconfiguration example of a file reproduction device corresponding to aninformation processing device according to an embodiment to which thepresent technology has been applied. A file reproduction device 200 inFIG. 22 is a device which reproduces MP4 files and MPD generated by thefile generation device 100 illustrated in FIG. 14 in the mannerdescribed above, generates a decoded image of either one or both of abase layer and an enhancement layer, and outputs the decoded image.

As illustrated in FIG. 22, the file reproduction device 200 includes anMPD analysis unit 201, an MP4 file reproduction unit 202, a base layerdecoding unit 203, and an enhancement layer decoding unit 204.

<Flow of MP4 File Reproduction Process>

The file reproduction device 200 illustrated in FIG. 22 performs an MP4file reproduction process to reproduce input MP4 files and generate adecoded image of an arbitrary layer. An example of a flow of the MP4file reproduction process is now described with reference to a flowchartshown in FIG. 23.

With a start of the MP4 file reproduction process, the MP4 filereproduction unit 202 determines whether to reproduce the base layer instep S171. When it is determined that the base layer is to bereproduced, the process proceeds to step S172.

In step S172, the MP4 file reproduction unit 202 acquires an MP4 file ofan extractor of the base layer.

In step S173, the MP4 file reproduction unit 202 acquires MP4 files oftiles to be reproduced on the basis of a track reference of the MP4 fileacquired in the previous step.

In step S174, the MP4 file reproduction unit 202 extracts encoded dataof a sample of a processing target from the acquired MP4 files.

In step S175, the base layer decoding unit 203 decodes the encoded datato generate a decoded image of the base layer (BL image).

In step S176, the base layer decoding unit 203 outputs the decoded imageof the base layer (BL image). After completion of processing in stepS176, the process proceeds to step S189.

On the other hand, when it is determined in step S171 that theenhancement layer is to be decoded, the process proceeds to step S178.

In step S178, the MP4 file reproduction unit 202 determines whether ornot the base layer has been acquired. When it is determined that thebase layer has not been acquired yet, the process proceeds to step S179.

Processing from step S179 to step S183 is performed similarly toprocessing from step S172 to step S176. When a decoded image of the baselayer (BL image) is output after completion of the foregoing processing,the process proceeds to step S184. On the other hand, when it isdetermined in step S178 that an image of the base layer has beenacquired, the process proceeds to step S184.

In step S184, the MP4 file reproduction unit 202 acquires an MP4 file ofan extractor of the enhancement layer.

In step S185, the MP4 file reproduction unit 202 acquires MP4 files oftiles to be reproduced on the basis of a track reference of the MP4 fileacquired in the previous step.

In step S186, the MP4 file reproduction unit 202 extracts encoded dataof a sample of a processing target from the acquired MP4 file.

In step S187, the enhancement layer decoding unit 204 performsprediction between layers by using a reference image of the base layerand encoding information as necessary to decode the encoded data, andgenerates a decoded image of the enhancement layer (EL image).

In step S188, the enhancement layer decoding unit 204 outputs thedecoded image of the enhancement layer (EL image). After completion ofprocessing in step S188, the process proceeds to step S189.

In step S189, the MP4 file reproduction unit 202 determines whether toend reproduction. When it is determined that reproduction is not to end,the process returns to step S171 to repeat processing in step S171 andsubsequent steps.

When it is determined in step S189 that reproduction is to end aftercompletion of processing from S171 to S189 in an appropriate manner foreach sample, the MP4 file reproduction process ends.

The file reproduction device 200 reproduces MP4 files having theconfiguration illustrated in FIGS. 15 and 16, and allows control ofdistribution for each tile in the enhancement layer as well as in thebase layer by performing the MP4 file reproduction process describedabove. Accordingly, a partial image (such as tile) of an image in eachlayer of an image constituted by a plurality of layers is adaptivelyprovided in image distribution.

<Flow of MPD Reproduction Process>

The file reproduction device 200 illustrated in FIG. 22 performs an MPDreproduction process to reproduce input MPD and generate a decoded imageof an arbitrary layer. A flow of the MPD reproduction process is nowdescribed with reference to a flowchart shown in FIG. 24.

With a start of the MPD reproduction process, the MPD analysis unit 201acquires MPD in step S201.

In step S202, the MPD analysis unit 201 determines whether to reproducea base layer. When it is determined that the base layer is to bereproduced, the process proceeds to step S203.

In step S203, the MPD analysis unit 201 specifies an adaptation set of atile of the base layer to be reproduced.

In step S204, the MPD analysis unit 201 allows the MP4 file reproductionunit 202 to acquire an MP4 file designated by a segment belonging to thespecified adaptation set.

In step 3205, the MP4 file reproduction unit 202 and the base layerdecoding unit 203 reproduce the acquired MP4 file to obtain a decodedimage of a tile of the base layer. According to this process, theprocessing from step S172 to step S176 in FIG. 23 is performed. Aftercompletion of processing in step S205, the process proceeds to stepS211.

On the other hand, when it is determined in step S202 that anenhancement layer is to be reproduced, the process proceeds to stepS206.

Processing in step S206 and step S207 is performed similarly to theprocessing in step S203 and step S204.

In step S208, the MPD analysis unit 201 specifies an adaptation set of atile of the enhancement layer to be reproduced.

In step S209, the MPD analysis unit 201 allows the MP4 file reproductionunit 202 to acquire an MP4 file designated by a segment belonging to thespecified adaptation set.

In step S210, the MP4 file reproduction unit 202 to the enhancementlayer decoding unit 204 reproduce the acquired MP4 file to obtain adecoded image of a tile of the enhancement layer. According to thisprocessing, the processing from step S178 to step S188 in FIG. 23 isperformed. After completion of processing in step S210, the processproceeds to step S211.

In step S211, the MPD analysis unit 201 determines whether to endreproduction. When it is determined that reproduction is not to end, theprocess returns to step S202 to repeat processing in step S202 andsubsequent steps.

When it is determined in step S211 that reproduction is to end aftercompletion of processing from step S202 to step S211 in an appropriatemanner for each sample, the MPD reproduction process ends.

The file reproduction device 200 reproduces MPD having a configurationillustrated in FIGS. 18 and 19, and allows control of distribution foreach tile in the enhancement layer as well as in the base layer byperforming the MPD reproduction process as described above. Accordingly,a partial image (such as tile) of an image in each layer of an imageconstituted by a plurality of layers is adaptively provided in imagedistribution.

2. Second Embodiment

<Reference to Base Layer>

A configuration for reference from each tile of an enhancement layer toeach tile of a base layer may be adopted. FIGS. 25 and 26 illustrate adifferent configuration example of MP4 files.

In this case, a track reference for reference to a base layer is addedto the track of each tile of the enhancement layer in the configurationillustrated in FIGS. 15 and 16.

For example, a track reference (Track Reference) of a track 7 in FIG. 26indicates reference to the track 2 by a reference type of “sbas”(sbas=2). A track reference (Track Reference) of a track 8 indicatesreference to the track 3 by a reference type “sbas” (sbas=3). A trackreference (Track Reference) of a track 9 indicates reference to thetrack 4 by a reference type “sbas” (sbas=4). A track reference (TrackReference) of a track 10 indicates reference to the track 5 by areference type “sbas” (sbas=5).

Accordingly, this configuration allows reference to tracks ofcorresponding tiles of the base layer from respective tracks storingtiles of the enhancement layer.

<Flow of MP4 File Generation Process>

An example of a flow of an MP4 file generation process in thisconfiguration example is now described with reference to a flowchartshown in FIG. 27.

Processing from step S221 to step S236 in FIG. 27 is performed similarlyto the processing from step S101 to step S116 in FIG. 17.

In step S237, the MP4 file generation unit 103 generates a trackreference indicating a track of a corresponding tile of the base layerin a track of each tile of the enhancement layer.

The processing from step S238 and step S239 in FIG. 27 is performedsimilarly to the processing from step S117 and step S118 in FIG. 17.

The file generation device 100 generates MP4 files having theconfiguration illustrated in FIGS. 25 and 26, and allows control ofdistribution for each tile in the enhancement layer as well as in thebase layer by performing the MP4 file generation process describedabove. Accordingly, a partial image (such as tile) of an image in eachlayer of an image constituted by a plurality of layers is adaptivelyprovided in image distribution.

<Configuration Example of MPD>

The MPD generation unit 104 of the file generation device 100 generatesMPD having a configuration illustrated in FIGS. 28 and 29, for example,from MP4 files generated by the MP4 file generation unit 103 in themanner described above.

In this case, association information for reference to the base layer isadded to a representation of each tile of an enhancement layer in theconfiguration illustrated in FIGS. 18 and 19.

For example, association information shown below for indicating areference relationship with an adaptation set of a tile 1 of a baselayer is set in a representation of a tile 1 of the enhancement layerillustrated in FIG. 29.

<associationid=“bl_tile1”associationType=“sbas”>

Similarly, association information shown below for indicating areference relationship with an adaptation set of a tile 2 of the baselayer is set in a representation of a tile 2 of the enhancement layer.

<associationid=“bl_tile2”associationType=“sbas”>

Similarly, association information shown below for indicating areference relationship with an adaptation set of a tile 3 of the baselayer is set in a representation of a tile 3 of the enhancement layer.

<associationid:=“bl_tile3”associationType=“sbas”>

Similarly, association information shown below for indicating areference relationship with an adaptation set of a tile 4 of the baselayer is set in a representation of a tile 4 of the enhancement layer.

<associationid=“bl_tile4”associationType=“sbas”>

Accordingly, this configuration allows reference to an adaptation set ofa corresponding tile of the base layer from an adaptation set of a tileof the enhancement layer.

<Flow of MPD Generation Process>

An example of a flow of this MPD generation process is now describedwith reference to a flowchart shown in FIGS. 30 and 31.

Processing from step S251 to step S266 in FIG. 30 is performed similarlyto the processing from step S121 to step S136 in FIG. 20.

In addition, processing from step S271 to step S282 in FIG. 31 isperformed similarly to the processing from step S141 to step S152 inFIG. 21.

In step S283 in FIG. 31, the MPD generation unit 104 sets associationinformation with a corresponding tile of a base layer in arepresentation of each tile of an enhancement layer.

Processing from step S284 to step S286 in FIG. 31 is performed similarlyto the processing from step S153 to step S155 in FIG. 21.

The file generation device 100 generates MPD having the configurationillustrated in FIGS. 28 and 29, and allows control of distribution foreach tile in the enhancement layer as well as in the base layer byperforming the MPD generation process described above. Accordingly, apartial image (such as tile) of an image in each layer of an imageconstituted by a plurality of layers is adaptively provided in imagedistribution.

Note that the MP4 file reproduction process and the MPD reproductionprocess, which are performed similarly to the corresponding processes inthe first embodiment, are not repeatedly explained herein.

3. Third Embodiment

<Filing for Each Layer>

FIGS. 32 and 33 illustrate a different configuration example of MP4files. As illustrated in these figures, MP4 files may be filed for eachlayer, for example. According to the example illustrated in FIG. 32,respective tracks storing information on a base layer (tracks 1 through5) are stored in one MP4 file. On the other hand, in case of the exampleillustrated in FIG. 33, respective tracks storing information on anenhancement layer (tracks 6 through 10) are stored in one MP4 file. Aninternal configuration in each track is similar to the correspondingconfiguration in the second embodiment (FIGS. 25 and 26). Needless tosay, an internal configuration in each track may have a configurationsimilar to the corresponding configuration in the first embodiment(FIGS. 15 and 16).

<Flow of MP4 File Generation Process>

An example of a flow of an MP4 file generation process in thisconfiguration example is now described with reference to a flowchartshown in FIG. 34.

Processing in step S301 is performed similarly to the processing in stepS221 in FIG. 27.

In step S302, the MP4 file generation unit 103 generates a track of anextractor of a base layer.

Processing in step S303 is performed similarly to the processing in stepS223 in FIG. 27.

In step S304, the MP4 file generation unit. 103 generates a track ofeach tile of the base layer.

Processing from step S305 to step S308 is performed similarly to theprocessing from step S225 to step S228 in FIG. 27.

In step S309, the MP4 file generation unit 103 collectively files therespective tracks of the base layer. In other words, the MP4 filegeneration unit 103 stores the respective tracks of the base layer inone file.

Processing in step S310 is performed similarly to the processing in stepS229 in FIG. 27.

In step S311, the MP4 file generation unit 103 generates a track of anextractor of an enhancement layer.

Processing in step S312 is performed similarly to the processing in stepS231 in FIG. 27.

In step S313, the MP4 file generation unit 103 generates a track of eachtile of the enhancement layer.

Processing from step S314 to step S319 is performed similarly to theprocessing from step S233 to step S238 in FIG. 27.

In step S320, the MP4 file generation unit 103 collectively files therespective tracks of the enhancement layer. In other words, the MP4 filegeneration unit 103 stores the respective tracks of the enhancementlayer in one file.

Processing in step S321 is performed similarly to the processing in stepS239 in FIG. 27.

The file generation device 100 generates MP4 files having theconfiguration illustrated in FIGS. 32 and 33, and allows control ofdistribution for each tile in the enhancement layer as well as in thebase layer by performing the MP4 file generation process describedabove. Accordingly, a partial image (such as tile) of an image in eachlayer of an image constituted by a plurality of layers is adaptivelyprovided in image distribution.

<Configuration Example of MPD>

The MPD generation unit 104 of the file generation device 100 generatesMPD having a configuration illustrated in FIGS. 35 and 36, for example,from MP4 files generated by the MP4 file generation unit 103 in themanner described above.

In this case, the MP4 files are filed for each layer, wherefore anadaptation set is generated for each layer in MPD.

According to an adaptation set of a base layer, a representationcontaining information on the whole of the base layer is set in an orderbelow the adaptation set as illustrated in FIG. 35. An MP4 file of thebase layer (bl.mp4) is set (registered) in a segment in an order belowthe representation. Information on each tile is set in a subrepresentation provided in an order below the representation.

A configuration of an enhancement layer illustrated in FIG. 36 issimilar to the configuration illustrated in FIG. 35.

<Flow of MPD Generation Process>

An example of a flow of an MPD generation process in this configurationexample is now described with reference to a flowchart shown in FIG. 37.

Processing from step S341 to step S343 is performed similarly to theprocessing from step S251 to step S253 in FIG. 30.

In step S344, the MPD generation unit 104 generates an adaptation set ofa base layer.

In step S345, the MPD generation unit 104 generates a representation ofthe base layer.

In step S346, the MPD generation unit 104 generates a segment of thebase layer.

In step S347, the MPD generation unit 104 generates a sub representationof each tile of the base layer.

In step S348, the MPD generation unit 104 sets encoding information inthe sub representation of each tile of the base layer.

In step S349, the MPD generation unit 104 sets level information in thesub representation of each tile of the base layer.

In step S350, the MPD generation unit 104 sets an essential property inthe sub representation of each tile of the base layer.

In step S351, the MPD generation unit 104 generates an adaptation set ofan enhancement layer.

In step S352, the MPD generation unit 104 generates a representation ofthe enhancement layer.

In step S353, the MPD generation unit 104 generates a segment of theenhancement layer.

In step S354, the MPD generation unit 104 generates a sub representationof each tile of the enhancement layer.

In step S355, the MPD generation unit 104 sets encoding information inthe sub representation of each tile of the enhancement layer.

In step S356, the MPD generation unit 104 sets level information in thesub representation of each tile of the enhancement layer.

In step S357, the MPD generation unit 104 sets an essential property inthe sub representation of each tile of the enhancement layer.

In step S358, the MPD generation unit 104 outputs the MPD thusgenerated.

The file generation device 100 generates MPD having the configurationillustrated in FIGS. 35 and 36, and allows control of distribution foreach tile in the enhancement layer as well as in the base layer byperforming the MPD generation process described above. Accordingly, apartial image (such as tile) of an image in each layer of an imageconstituted by a plurality of layers is adaptively provided in imagedistribution.

<Flow of MP4 File Reproduction Process>

An example of a flow of an MP4 file reproduction process in theforegoing configuration example is now described with reference to aflowchart shown in FIG. 38.

With a start of the MP4 file reproduction process, the MP4 filereproduction unit 202 determines whether to reproduce a base layer instep S371. When it is determined that the base layer is to bereproduced, the process proceeds to step S372.

In step S372, the MP4 file reproduction unit 202 acquires MP4 files ofthe base layer.

In step S373, the MP4 file reproduction unit 202 specifies a track of atile to be reproduced on the basis of a track reference of the acquiredMP4 files.

In step S374, the MP4 file reproduction unit 202 extracts encoded dataon a sample of a processing target from the specified track.

In step S375, the base layer decoding unit 203 decodes the encoded datato generate a decoded image of the base layer (BL image).

In step S376, the base layer decoding unit 203 outputs the decoded imageof the base layer (BL image). After completion of processing in stepS376, the process proceeds to step S388.

On the other hand, when it is determined in step S371 that anenhancement layer is to be decoded, the process proceeds to step S377.

In step S377, the MP4 file reproduction unit 202 determines whether ornot the base layer has been acquired. When it is determined that thebase layer has not been acquired yet, the process proceeds to step S378.

Processing from step S378 to step S382 is performed similarly to theprocessing from step S372 to step S376. When a decoded image of the baselayer (BL image) is output after completion of the processing of thesesteps, the process proceeds to step S383. On the other hand, when it isdetermined in step S377 that an image of the base layer has beenacquired, the process proceeds to step S383.

In step S383, the MP4 file reproduction unit 202 acquires MP4 files ofthe enhancement layer.

In step S384, the MP4 file reproduction unit 202 acquires a track of arile to be reproduced on the basis of a track reference of the acquiredMP4 files.

In step S385, the MP4 file reproduction unit 202 extracts encoded dataof a sample of a processing target from the specified track.

In step S386, the enhancement layer decoding unit 204 performsprediction between layers by using a reference image of the base layerand encoding information as necessary, and decodes the encoded data togenerate a decoded image of the enhancement layer (EL image).

In step S387, the enhancement layer decoding unit 204 outputs thedecoded image of the enhancement layer (EL image. After completion ofprocessing in step S387, the process proceeds to step S388.

In step S388, the MP4 file reproduction unit 202 determines whether toend reproduction. When it is determined that reproduction is not to end,the process returns to step S371 to repeat processing in seep S371 andsubsequent steps.

When it is determined in step S388 that reproduction is to end aftercompletion of processing from step S371 to step S388 in an appropriatemanner for each sample, the MP4 file reproduction process ends.

The file reproduction device 200 reproduces MP4 files having theconfiguration illustrated in FIGS. 32 and 33, and allows control ofdistribution for each tile in the enhancement layer as well as in thebase layer by performing the MP4 file reproduction process describedabove. Accordingly, a partial image (such as tile) of an image in eachlayer of an image constituted by a plurality of layers is adaptivelyprovided in image distribution.

<Flow of MPD Reproduction Process>

An example of a flow of an MPD reproduction process in the foregoingconfiguration example is now described with reference to a flowchartshown in FIG. 39.

With a start of the MPD reproduction process, the MPD analysis unit 201acquires MPD in step S401.

In step 3402, the MPD analysis unit 201 determines whether to reproducea base layer. When it is determined that the base layer is to bereproduced, the process proceeds to step 3403.

In step S403, the MPD analysis unit 201 specifies an adaptation set ofthe base layer.

In step S404, the MPD analysis unit 201 allows the MP4 file reproductionunit 202 to acquire an MP4 file designated by a segment belonging to thespecified adaptation set.

In step S405, the MPD analysis unit 201 specifies a sub representationof a tile to be reproduced.

In step S406, the MP4 file reproduction unit 202 and the base layerdecoding unit 203 reproduce a track corresponding to the specified subrepresentation of the acquired MP4 file to obtain a decoded image of atile of the base layer. According to this process, the processing fromstep S372 to step S376 in FIG. 38 is performed. After completion of thestep S406, the process proceeds to step S414.

On the other hand, when it is determined in step S402 that anenhancement layer is to be reproduced, the process proceeds to stepS407.

Processing from step S407 to step S409 is performed similarly to theprocessing from step S403 and step S405.

In step S410, the MPD analysis unit 201 specifies an adaptation set ofthe enhancement layer.

In step S411, the MPD analysis unit 201 allows the MP4 file reproductionunit 202 to acquire an MP4 file designated by a segment belonging to thespecified adaptation set.

In step S412, the MPD analysis unit 201 specifies a sub representationof a tile to be reproduced.

In step S413, the MP4 file reproduction unit 202 to the enhancementlayer decoding unit 204 reproduce a track corresponding to the specifiedsub presentation of the acquired MP4 file to obtain a decoded image of atile of the enhancement layer. According to this process, the processingfrom step S377 to step S387 in FIG. 38 is performed. After completion ofprocessing in step S413, the process proceeds to step S414.

In step S414, the MPD analysis unit 201 determines whether to endreproduction. When it is determined that reproduction is not to end, theprocess returns to step S402 to repeat processing in step S402 andsubsequent steps.

When it is determined in step S414 that reproduction is to end aftercompletion of processing from step S402 to step S414 in an appropriatemanner for each sample, the MPD reproduction process ends.

The file reproduction device 200 reproduces MPD having the configurationillustrated in FIGS. 35 and 36, and allows control of distribution foreach tile in the enhancement layer as well as in the base layer byperforming the MPD reproduction process described above. Accordingly, apartial image (such as tile) of an image in each layer of an imageconstituted by a plurality of layers is adaptively provided in imagedistribution.

4. Fourth Embodiment

<Filing of all Layers>

FIGS. 40 and 41 illustrate a different configuration example of an MP4file. As illustrated in these figures, all layers of an MP4 file may befiled as one file. An internal configuration in each track is similar tothe corresponding configuration in the second embodiment (FIGS. 25 and26). Needless to say, an internal configuration in each track may have aconfiguration similar to the corresponding configuration in the firstembodiment (FIGS. 15 and 16).

<Flow of MP4 File Generation Process>

An example of a flow of an MP4 file generation process in thisconfiguration example is now described with reference to a flowchartshown in FIG. 42.

Processing from step S431 to step S448 is performed similarly to theprocessing from step S301 to step S308 and the processing from step S310to step S319 in FIG. 34.

In step S449, the MP4 file generation unit 103 collectively filesrespective tiles (respective tracks) of all layers (generates one MP4file containing all tracks).

In step S450, the MP4 file generation unit 103 outputs the generated MP4file.

The file generation device 100 generates MP4 files having theconfiguration illustrated in FIGS. 40 and 41, and allows control ofdistribution for each tile in the enhancement layer as well as in thebase layer by performing the MP4 file generation process describedabove. Accordingly, a partial image (such as tile) of an image in eachlayer of an image constituted by a plurality of layers is adaptivelyprovided in image distribution.

<Configuration Example of MPD>

The MPD generation unit 104 of the file generation device 100 generatesMPD having a configuration illustrated in FIGS. 43 and 44, for example,from an MP4 file generated by the MP4 file generation unit 103 in themanner described above.

In this case, only one adaptation set is generated (adaptation setcommon to all layers is generated) as illustrated in FIGS. 43 and 44.

A representation containing information on all layers is generated in anorder below the adaptation set. An MP4 file of all the layers (ble1.mp4)is set (registered) in a segment in an order below the representation.Information on each tile of each layer is set in a sub representation inan order lower than the representation.

<Flow of MPD Generation Process>

An example of a flow of an MPD generation process in this configurationexample is now described with reference to a flowchart shown in FIG. 45.

Processing from step S471 to step S473 is performed similarly to theprocessing from step S251 to step S253 in FIG. 30.

In step S474, the MPD generation unit 104 generates an adaptation setcommon to all the layers.

In step S475, the MPD generation unit 104 generates a representationcommon to all the layers.

In step S476, the MPD generation unit 104 generates a segment common toall the layers.

In step S477, the MPD generation unit 104 generates a sub representationof each tile of each layer.

In step S478, the MPD generation unit 104 sets encoding information inthe sub representation of each tile of each layer.

In step S479, the MPD generation unit 104 sets level information in thesub representation of each tile of each layer.

In step S480, the MPD generation unit 104 sets an essential property inthe sub representation of each tile of each layer.

In step S481, the MPD generation unit 104 outputs MPD thus generated.

The file generation device 100 generates MPD having the configurationillustrated in FIGS. 43 and 44, and allows control of distribution foreach tile in the enhancement layer as well as in the base layer byperforming the MPD generation process described above. Accordingly, apartial image (such as tile) of an image in each layer of an imageconstituted by a plurality of layers is adaptively provided in imagedistribution.

<Flow of MP4 File Reproduction Process>

An example of a flow of an MP4 file reproduction process in theforegoing configuration example is now described with reference to aflowchart shown in FIG. 46.

With a start of the MP4 file reproduction process, the MP4 filereproduction unit 202 acquires an MP4 file in step S501.

In step S502, it is determined whether to reproduce a base layer. Whenit is determined that the base layer is to be reproduced, the processproceeds to step S503.

In step S503, the MP4 file reproduction unit 202 specifies a track of atile of the base layer to be reproduced on the basis of a trackreference of a track containing information indicating the whole of thebase layer.

Processing from step S504 to step S506 is performed similarly to theprocessing from step S374 to step S376 in FIG. 38.

After completion of processing in step S506, the process proceeds tostep S516.

On the other hand, when it is determined in step S502 that anenhancement layer is to be decoded, the process proceeds to step S507.

In step S507, the MP4 file reproduction unit 202 determines whether thebase layer has been acquired. When it is determined that the base layerhas not been acquired yet, the process proceeds to step S508.

Processing from step S508 to step S511 is performed similarly to theprocessing from step S503 to step S506. When a decoded image of the baselayer (BL image) is output after completion of processing of thesesteps, the process proceeds to step S512. On the other hand, when it isdetermined in step S507 that an image of the base layer has beenacquired, the process proceeds to step S512.

In step S512, the MP4 file reproduction unit 202 acquires a track of atile of the enhancement layer to be reproduced on the basis of a trackreference of a track containing information indicating the whole of theenhancement layer.

Processing from step S513 to step S515 is performed similarly to theprocessing from step S385 to step S387 in FIG. 38. After completion ofprocessing in step S515, the process proceeds to step S516.

In step S516, the MP4 file reproduction unit 202 determines whether toend reproduction. When it is determined that reproduction is not to end,the process returns to step S501 to repeat processing in step S501 andsubsequent steps.

When it is determined in step S516 that reproduction is to end aftercompletion of processing from step S501 to step S516 in an appropriatemanner for each sample, the MP4 file reproduction process ends.

The file reproduction device 200 reproduces MP4 files having theconfiguration illustrated in FIGS. 40 and 41, and allows control ofdistribution for each tile in the enhancement layer as well as in thebase layer by performing the MP4 file reproduction process describedabove. Accordingly, a partial image (such as tile) of an image in eachlayer of an image constituted by a plurality of layers is adaptivelyprovided in image distribution.

<Flow of MPD Reproduction Process>

An example of a flow of an MPD reproduction process in the foregoingconfiguration example is now described with reference to a flowchartshown in FIG. 47.

With a start of an MPD reproduction process, the MPD analysis unit 201acquires MPD in step S531.

In step S532, the MPD analysis unit 201 allows the MP4 file reproductionunit 202 to acquire an MP4 file designated by a segment belonging to anadaptation set common to all layers of the acquired MPD.

In step S533, the MPD analysis unit 201 determines whether to reproducea base layer. When it is determined that the base layer is to bereproduced, the process proceeds to step S534.

In step S534, the MPD analysis unit 201 specifies a sub representationof a tile to be reproduced.

In step S535, the MP4 file reproduction unit 202 and the base layerdecoding unit 203 reproduce a track corresponding to the specified subrepresentation of the acquired MP4 file to obtain a decoded image of atile of the base layer. According to this process, processing from stepS503 to step S506 in FIG. 46 is performed. After completion ofprocessing in step S535, the process proceeds to step S538.

When it is determined in step S533 that an enhancement layer is to bereproduced, the process proceeds to step S536.

In step S536, the MPD analysis unit 201 specifies a sub representationof a tile to be reproduced.

In step S537, the MP4 file reproduction unit 202 to the enhancementlayer decoding unit 204 reproduce a track corresponding to the specifiedsub presentation of the acquired MP4 file to obtain a decoded image of atile of the enhancement layer. According to this process, processingfrom step S507 to step S515 in FIG. 46 is performed. After completion ofprocessing in step S537, the process proceeds to step S538.

In step S538, the MPD analysis unit 201 determines whether to endreproduction. When it is determined that reproduction is not to end, theprocess returns to step S531 to repeat processing in step S531 andsubsequent steps.

When it is determined in step S538 that reproduction is to end aftercompletion of processing from step S531 to step S538 in an appropriatemanner for each sample, the MPD reproduction process ends.

The file reproduction device 200 reproduces MPD having the configurationillustrated in FIGS. 43 and 44, and allows control of distribution foreach tile in the enhancement layer as well as in the base layer byperforming the MPD reproduction process described above. Accordingly, apartial image (such as tile) of an image in each layer of an imageconstituted by a plurality of layers is adaptively provided in imagedistribution.

5. Fifth Embodiment

<Distribution System>

The respective device described in the foregoing embodiments isapplicable to a distribution system which distributes still images andmoving images, for example. This application example is hereinafterdescribed.

FIG. 48 is a view illustrating a general configuration example of adistribution system to which the present technology has been applied. Adistribution system 300 illustrated in FIG. 48 is a system whichdistributes still images and moving images. As illustrated in FIG. 48,the distribution system 300 includes a distribution data generationdevice 301, a distribution server 302, a network 303, a terminal device304, and a terminal device 305.

The distribution data generation device 301 generates distribution datain a distribution format on the basis of data on distributed stillimages and moving images. The distribution data generation device 301supplies generated distribution data to the distribution server 302. Thedistribution server 302 stores distribution data generated by thedistribution data generation device 301 in a storage unit or the like tomanage the data, and provides distribution services of the distributiondata to the terminal device 304 and the terminal device 305 via thenetwork 303.

The network 303 is a communication network corresponding to acommunication medium. The network 303 may be constituted by an arbitrarycommunication network, such as a wired communication network, a wirelesscommunication network, and a combination of both. For example, thenetwork 303 may be a wired local area network (LAN), a wireless LAN, apublic telephone line network, a broadband communication network forwireless mobiles such as so-called 3G line and 4G line, the Internet, ora combination of these networks. Moreover, the network 303 may beconstituted by a single communication network, or a plurality ofcommunication networks. Furthermore, a part or the whole of the network303 may be constituted by a communication cable in conformity topredetermined standards, such as a universal serial bus (USB) cable anda high-definition multimedia interface (HDMI, registered trademark)cable.

The distribution server 302, the terminal device 304, and the terminaldevice 305 are connected to the network 303 in a state communicativewith each other. A method for connecting these devices to the network303 may be arbitrarily determined. For example, these devices may beconnected to the network 303 via wired communication or wirelesscommunication. In addition, for example, these devices may be connectedto the network 303 via arbitrary communication devices (communicationfacilities), such as access points, relay devices, and base stations.

Each of the terminal device 304 and the terminal device 305 isconstituted by an arbitrary electronic device having a communicationfunction, such as a cellular phone, a smartphone, a tablet-typecomputer, and a note-type computer. Each of the terminal device 304 andthe terminal device 305 requests the distribution server 302 todistribute a distribution file on the basis of an instruction issuedfrom a user or the like, for example.

The distribution server 302 transmits requested distribution data to arequest source. The terminal device 304 or the terminal device 305having requested distribution receives and reproduces the distributiondata.

According to the distribution system 300 having this configuration, thepresent technology described in the respective embodiments is applied tothe distribution data generation device 301. In other words, the filegeneration device 100 described above is applied to the distributiondata generation device 301.

In addition, the present technology described in the respectiveembodiments is applied to the terminal device 304 and the terminaldevice 305. In other words, the file reproduction device 200 describedabove is applied to the terminal device 304 and the terminal device 305.

According to this configuration, the distribution data generation device301, the terminal device 304, and the terminal device 305 offer effectssimilar to the effects of the foregoing respective embodiments. Morespecifically, the distribution system 300 is capable of adaptivelyproviding a partial image of an image in each layer of an imageconstituted by a plurality of layers, and realizing switching of layersas described in the use case shown in the first embodiment, for example.

6. Sixth Embodiment

<Computer>

The foregoing series of processes may be executed either by hardware orby software. When the series of processes is executed by software,programs constituting the software are installed into a computer. Thecomputer in this context includes a computer incorporated into dedicatedhardware, and a computer capable of executing various types of functionsunder various types of programs installed into the computer, such as ageneral-purpose personal computer.

FIG. 49 is a block diagram illustrating a configuration example ofhardware of a computer which executes the series of processes describedabove under programs.

According to a computer 400 illustrated in FIG. 49, a central processingunit (CPU) 401, a read only memory (ROM) 402, and a random access memory(RAM) 403 are connected to each other via a bus 404.

An input/output interface 410 is further connected to the bus 404. Aninput unit 411, an output unit 412, a storage unit 413, a communicationunit 414, and a drive 415 are connected to the input/output interface410.

The input unit 411 is constituted by a keyboard, a mouse, a microphone,a touch panel, an input terminal or the like. The output unit 412 isconstituted by a display, a speaker, an output terminal or the like. Thestorage unit 413 is constituted by a hard disk, a RAM disk, anon-volatile memory or the like. The communication unit 414 isconstituted by a network interface or the like. The drive 415 drives aremovable medium 421 such as a magnetic disk, an optical disk, amagneto-optical disk, and a semiconductor memory.

According to the computer having this configuration, the CPU 401 loadsprograms stored in the storage unit 413 into the RAM 403 via theinput/output interface 410 and the bus 404, and executes the programs toperform the series of processes described above, for example. Data orthe like necessary for execution of the respective processes by the CPU401 is also appropriately stored in the RAM 403.

The programs executed by the computer (CPU 401) may be recorded in theremovable medium 421 constituting a package medium, for example. In thiscase, the programs are installed into the storage unit 413 via theinput/output interface 410 in a state of attachment of the removablemedium 421 to the drive 415.

Alternatively, the programs may be provided via a wired or wirelesstransmission medium such as a local area network, the Internet, anddigital satellite broadcasting. In this case, the programs may bereceived by the communication unit 414 and installed into the storageunit 413.

Instead, the programs may be pre-installed in the ROM 402 or the storageunit 413.

Note that the programs executed by the computer may be programs underwhich processes are executed in time series in the order described inthe present specification, or executed in parallel or at necessarytiming such as on occasions of calls.

The steps describing the programs recorded in a recording medium containnot only processes executed in time series in the order discussedherein, but also processes executed in parallel or individually, ratherthan executed in time series.

Processes performed by the respective steps may be executed by theforegoing respective devices, or arbitrary devices other than theforegoing devices. In this case, the devices executing the processeshave functions (function blocks or the like) necessary for executing theprocesses described above. In addition, information necessary for theprocesses is appropriately transmitted to the devices.

According to the present specification, the system refers to acollection of multiple constituent elements (such as devices and modules(parts)), and includes both cases where all the constituent elements arecontained in the same housing, and where some of the constituentelements are not contained in the same housing. Accordingly, multipledevices accommodated in separate housings and connected via a network,and one device including multiple modules accommodated within onehousing are both regarded as systems.

According to the foregoing description, a configuration discussed as onedevice (or processing unit) may be divided into multiple devices (orprocessing units). On the contrary, the configuration discussed asmultiple devices (or processing units) may be combined into one device(or processing unit). Needless to say, configurations not discussedherein may be added to the configurations of the respective devices (orrespective processing units). In addition, when the configuration andoperation of the whole system are substantially identical, a part of aconfiguration of a certain device (or processing unit) may beincorporated into a configuration of another device (or anotherprocessing unit).

While the preferred embodiments according to the present disclosure havebeen described in detail with reference to the accompanying drawings,the technical scope of the present disclosure is not limited to theseexamples. It is obvious that various examples of changes andmodifications may be presented by those having ordinary knowledge in thetechnical field of the present disclosure in the light of the presenttechnology within the scope of the technical spirit described in theclaims. It is therefore understood that these changes and modificationsare also contained in the technical range of the present disclosure as amatter of course.

For example, the present technology is applicable to a system of cloudcomputing where one function is shared by multiple devices and processedin cooperation with one another via a network.

Moreover, the respective steps discussed with reference to the foregoingflowcharts may be shared and executed by multiple devices rather thanexecuted by one device.

Furthermore, when multiple processes are contained in one step, themultiple processes contained in the one step may be shared and executedby multiple devices rather than executed by one device.

Alternatively, the present technology may be practiced in the form ofarbitrary configurations mounted on the foregoing device or devicesconstituting the foregoing system, such as a processor as a system largescale integration (LSI), a module including a plurality of processors, aunit including a plurality of modules, and a set including functions inaddition to functions of a unit (i.e., configuration of a part of adevice).

The present technology may have the following configurations.

(1) An information processing device comprising a file generation unitthat generates a file that stores information on a whole of a base layerof encoded data produced by layer encoding of image data, information oneach of partial areas of an image of the base layer, information on awhole of an enhancement layer of the encoded data, and information oneach of partial areas of an image of the enhancement layer such thateach of the information is stored in corresponding one of tracksdifferent from each other.(2) The information processing device according to (1), wherein

the file generation unit stores, in the track storing the information onthe whole of the base layer, encoding information indicating an encodingsystem of the base layer,

the file generation unit stores, in each of the tracks storing theinformation on the partial areas of the base layer, encoding informationindicating that the corresponding track stores only the information onthe corresponding partial area of the base layer,

the file generation unit stores, in the track storing the information onthe whole of the enhancement layer, encoding information indicating anencoding system of the enhancement layer, and

the file generation unit stores, in each of the tracks storing theinformation on the partial areas of the enhancement layer, encodinginformation indicating that the corresponding track stores only theinformation on the corresponding partial area of the enhancement layer.

(3) The information processing device according to (1) or (2), wherein

the file generation unit stores, in the track storing the information onthe whole of the base layer, information indicating reference to thetracks storing the information on the partial areas of the base layer,and information indicating a type of the reference,

the file generation unit stores, in each of the tracks storing theinformation on the partial areas of the base layer, informationindicating reference to the track storing the information on the wholeof the base layer, and information indicating a type of the reference,

the file generation unit stores, in the track storing the information onthe whole of the enhancement layer, information indicating reference tothe track storing the information on the whole of the base layer andinformation indicating a type of the reference, and informationindicating reference to the tracks storing the information on thepartial areas of the enhancement layer and information indicating a typeof the reference, and

the file generation unit stores, in each of the tracks storinginformation on the partial areas of the enhancement layer, informationindicating reference to the track storing the information on the wholeof the enhancement layer, and information indicating a type of thereference.

(4) The information processing device according to (3), wherein the filegeneration unit further stores, in each of the tracks storing theinformation on the partial areas of the enhancement layer, informationindicating reference to the track storing the information on the partialarea of the base layer in correspondence with the corresponding partialarea of the enhancement layer.(5) The information processing device according any one of (1) through(4), wherein

the file generation unit stores, in each of the tracks storing theinformation on the whole of the layers, a sample of referenceinformation for the encoded data on

the partial areas of the corresponding layer, and the file generationunit stores, in each of the tracks storing the information on thepartial areas of the layers, a sample of the encoded data on thecorresponding partial area.

(6) The information processing device according to any one of (1)through (5), wherein the file generation unit stores, in each of thetracks storing the information on the partial areas of the layers,position information indicating a position of the corresponding partialarea.(7) The information processing device according to any one of (1)through (6), wherein the file generation unit generates the file foreach of the tracks.(8) The information processing device according to any one of (1)through (7), wherein the file generation unit generates the file foreach of the layers.(9) The information processing device according to any one of (1)through (8), wherein the file generation unit generates a file storingall of the tracks.(10) An information processing method generating a file that storesinformation on a whole of a base layer of encoded data produced by layerencoding of image data, information on each of partial areas of an imageof the base layer, information on a whole of an enhancement layer of theencoded data, and information on each of partial areas of an image ofthe enhancement layer such that each of the information is stored incorresponding one of tracks different from each other.(11) An information processing device comprising a metadata generationunit that generates metadata that contains information on a whole of abase layer of encoded data produced by layer encoding of image data,information on each of partial areas of an image of the base layer,information on a whole of an enhancement layer of the encoded data, andinformation on each of partial areas of an image of the enhancementlayer.(12) The information processing device according to (11), wherein

the information storing information on the whole of the base layercontains encoding information indicating an encoding system of the baselayer,

the information on each of the partial areas of the base layer containsencoding information indicating that only the information on thecorresponding partial area of the base layer is contained,

the information on the whole of the enhancement layer contains encodinginformation indicating an encoding system of the enhancement layer, and

the information on each of the partial areas of the enhancement layercontains encoding information indicating that only the information onthe corresponding partial area of the enhancement layer is contained.

(13) The information processing device according to (11) or (12),wherein

the information on the whole of the base layer contains informationindicating reference to the information on the partial areas of the baselayer, and information indicating a type of the reference,

the information on each of the partial areas of the base layer containsinformation indicating reference to the information on the whole of thebase layer, and information indicating a type of the reference,

the information on the whole of the enhancement layer containsinformation indicating reference to the information on the whole of thebase layer and information indicating a type of the reference, andinformation indicating reference to the information on the partial areasof the enhancement layer and information indicating a type of thereference, and

the information on each of the partial areas of the enhancement layerscontains information indicating reference to the information on thewhole of the enhancement layer, and information indicating a type of thereference.

(14) The information processing device according to (13), wherein theinformation on each of the partial areas of the enhancement area furthercontains information indicating reference to the information on thepartial area of the base layer in correspondence with the correspondingpartial area of the enhancement layer, and information indicating a typeof the reference.(15) The information processing device according to any one of (11)through (14), wherein each of the information on the partial areas ofthe base layer, and the information on the partial areas of theenhancement layer contains position information indicating a position ofthe corresponding partial area.(16) The information processing device according any one of (11) through(15), wherein

the metadata generation unit sets, in the information on the whole ofthe base layer, a file that stores a sample of reference information forthe encoded data of the partial areas of the base layer,

the metadata generation unit sets, in the information on the partialareas of the base layer, a file that stores a sample of the encoded dataof the corresponding partial area of the base layer,

the metadata generation unit sets, in the information on the whole ofthe enhancement layer, a file that stores a sample of referenceinformation for the encoded data of the partial areas of the enhancementlayer, and

the metadata generation unit sets, in the information on the partialareas of the enhancement layer, a file that stores a sample of theencoded data of the corresponding partial area of the enhancement layer.

(17) The information processing device according to any one of (11)through (16), wherein

the metadata generation unit sets a file that stores the encoded data ofthe base layer in an order higher than the information on the whole ofthe base layer and the information on the partial areas of the baselayer, and

the metadata generation unit sets a file that stores the encoded data ofthe enhancement layer in an order higher than the information on thewhole of the enhancement layer and the information on the partial areasof the enhancement layer.

(18) The information processing device according any one of (11) through(17), wherein the metadata generation unit sets a file that stores theencoded data of all of the layers in an order higher than theinformation on the whole of the base layer, the information on thepartial areas of the base layer, the information on the whole of theenhancement layer, and the information on the partial areas of theenhancement layer.(19) The information processing device according to any one of (11)through (18), wherein the metadata generation unit sets the informationon the whole of the base layer, the information on each of the partialareas of an image of the base layer, the information on the whole of theenhancement layer, and the information on each of the partial areas ofan image of the enhancement layer such that each of the information isset in corresponding one of adaptation sets different from each other,or in corresponding one of sub representations different from eachother.(20) An information processing method generating metadata that containsinformation on a whole of a base layer of encoded data produced by layerencoding of image data, information on each of partial areas of an imageof the base layer, information on a whole of an enhancement layer of theencoded data, and information on each of partial areas of an image ofthe enhancement layer.

REFERENCE SIGNS LIST

-   100 File generation device-   101 Base layer encoding unit-   102 Enhancement layer encoding unit-   103 MP4 file generation unit-   104 MPD generation unit-   200 File reproduction device-   201 MPD analysis unit-   202 MP4 file reproduction unit-   203 Base layer decoding unit-   204 Enhancement layer decoding unit-   300 Distribution system-   301 Distribution data generation device-   302 Distribution server-   303 Network-   304 and 305 Terminal device-   400 Computer

The invention claimed is:
 1. An information processing device comprisinga file generation unit that generates a file that stores information ona whole of an enhancement layer of encoded data produced by layerencoding of image data, and information on each of partial areas of animage of the enhancement layer such that each of the information isstored in corresponding one of tracks different from each other.
 2. Theinformation processing device according to claim 1, wherein the filegeneration unit generates the file that further stores information on awhole of a base layer of the encoded data in corresponding one of thetracks different from each other.
 3. The information processing deviceaccording to claim 2, wherein the file generation unit generates thefile that further stores information on each of partial areas of animage of the base layer of the encoded data in corresponding one of thetracks different from each other.
 4. The information processing deviceaccording to claim 3, wherein the file generation unit stores, in thetrack storing the information on the whole of the base layer, encodinginformation indicating an encoding system of the base layer, the filegeneration unit stores, in each of the tracks storing the information onthe partial areas of the base layer, encoding information indicatingthat the corresponding track stores only the information on thecorresponding partial area of the base layer, the file generation unitstores, in the track storing the information on the whole of theenhancement layer, encoding information indicating an encoding system ofthe enhancement layer, and the file generation unit stores, in each ofthe tracks storing the information on the partial areas of theenhancement layer, encoding information indicating that thecorresponding track stores only the information on the correspondingpartial area of the enhancement layer.
 5. The information processingdevice according to claim 3, wherein the file generation unit stores, inthe track storing the information on the whole of the base layer,information indicating reference to the tracks storing the informationon the partial areas of the base layer, and information indicating atype of the reference, the file generation unit stores, in each of thetracks storing the information on the partial areas of the base layer,information indicating reference to the track storing the information onthe whole of the base layer, and information indicating a type of thereference, the file generation unit stores, in the track storing theinformation on the whole of the enhancement layer, informationindicating reference to the track storing the information on the wholeof the base layer and information indicating a type of the reference,and information indicating reference to the tracks storing theinformation on the partial areas of the enhancement layer andinformation indicating a type of the reference, and the file generationunit stores, in each of the tracks storing information on the partialareas of the enhancement layer, information indicating reference to thetrack storing the information on the whole of the enhancement layer, andinformation indicating a type of the reference.
 6. The informationprocessing device according to claim 5, wherein the file generation unitfurther stores, in each of the tracks storing the information on thepartial areas of the enhancement layer, information indicating referenceto the track storing the information on the partial area of the baselayer in correspondence with the corresponding partial area of theenhancement layer.
 7. The information processing device according toclaim 3, wherein the file generation unit stores, in each of the tracksstoring the information on the whole of the layers, a sample ofreference information for the encoded data on the partial areas of thecorresponding layer, and the file generation unit stores, in each of thetracks storing the information on the partial areas of the layers, asample of the encoded data on the corresponding partial area.
 8. Theinformation processing device according to claim 3, wherein the filegeneration unit stores, in each of the tracks storing the information onthe partial areas of the layers, position information indicating aposition of the corresponding partial area.
 9. The informationprocessing device according to claim 1, wherein the file generation unitgenerates the file for each of the tracks.
 10. The informationprocessing device according to claim 1, wherein the file generation unitgenerates the file for each of the layers.
 11. The informationprocessing device according to claim 1, wherein the file generation unitgenerates a file storing all of the tracks.
 12. An informationprocessing method generating a file that stores information on a wholeof an enhancement layer of encoded data produced by layer encoding ofimage data, and information on each of partial areas of an image of theenhancement layer such that each of the information is stored incorresponding one of tracks different from each other.